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^h'iP^S^ective  of  tipis  study  was  to  evaluate  Environmental  Stress  Screening  (ESS) 
teclinlques  contained  in  DOD-HDBK-344 by  applying  the  methodology  to  several  electronic 
products  during  actual  factory  production.  Validation  of  the  techniques,  the  develop¬ 
ment  of  Improved,  ^Im^lfied,  and  automated  procedures  and  subsequent  revisions  to  the 
Handbook  were  the  objectives,  of  the  evaluation.  The  Rome  Laboratory  has  developed 
techniques  which  enable  users  to  plan,  monitor,  and  control  ESS  PROGRAMS.  These 
techniques  were  previously  documented  in  DOt-HDBK-344 ,  "ESS  of  Electronic  Hardware." 

The  basic  approach  to  the  ESS  process  used  in  the  Handbook  focuses  on  the  latent  defect 
population  of  the  hardware,  and  on  developing  a  cost-effective,  statistically 
controlled  ESS  process,  aimed  at  reducing  the  defect  population  so  as  to  achieve 
quantitative  field  reliability  requirements.  As  a  result  of  the  study:  the  effective¬ 
ness  of  the  Handboole  methodology  and  techniques  was  confirmed  and  validated;  recommended 
changes,  additions,  and  deletions  were  Incorporated  into  a  revision  of  the  Handbook  and 
software  package  wa^ developed  to  facilitate  application  of  the  procedures.'  The  study 
has  also  shown  that  the  Handbook  procedures  provide  a  much  needed  linkage  to  TQM 
approaches,  wherein  customer  reliability  and  cost  requirements  are  translated  into  a 
suite  of  goals  that  ^an  be  measured  and  controlled  in  a  factory  manufacturing  environ- 

ment.  The  revised  Handbook  and  application/evaluation  results  are  (see  reverse) _ 
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FOREWORD 


Before  DOD  HDBK  344,  the  usual  practices  for  environmental  stress 
screening  (ESS)  involved  the  application  of  fixed  screening  regimens,  often 
defined  by  contract.  The  disadvantages  with  this  approach  to  ESS  were  that  i) 
it  was  not  known  whether  the  reliability  requirements  were  being  achieved 
and  thus  whether  or  not  the  ESS  was  inadequate  (or  conversely  too  excessive), 
ii)  there  was  no  provision  for  replacing  screening  with  defect  prevention  and 
demonstrated  process  control,  and  iii)  the  ESS  program  was  not  necessarily 
optimized  for  cost. 

With  the  introduction  of  DOD  HDBK  344,  the  application  of  ESS  was  changed 
from  the  predefined  approach  to  a  quantitati  *■:  approach  that  emphasized  the 
use  of  ESS  to  identify  and  therefore  eliminate  the  causes  of  field  defects,  and  to 
design  and  dynamically  modify  an  ESS  program  that  optimized  the  combined 
user  and  producer's  cost  of  achieving  the  desired  field  reliability.  The 
implemented  ESS  program  was  monitored  and  controlled  using  statistical 
process  control  techniques.  With  the  DOD  HDBK  344  quantitative  approach, 
the  ESS  was  thus  custom  tailored  to  suit  existing  requirements,  design,  and 
factory  capabilities.  The  approach  assumed  that  the  user  conducted  the 
necessary  surveys  and  studies  to  determine  the  actual  stress  levels  to  which 
the  equipment  was  subjected  and  provided  the  quantitative  approach  to 
determine  the  adequacy  and  efficacy  of  the  ESS  program. 

In  order  to  assure  that  the  DOD  HDBK  344  approach  and  methodologies  were 
practical  and  could  be  be  implemented  in  an  actual  factory  environment, 
RADC  commissioned  this  study  contract  to  apply  the  methodology  to  three 
equipment  types.  With  the  changes  recommended  in  this  report  and  reflected 
in  a  revised  HDBK,  the  study  confirmed  the  methodologies  of  DOD  HDBK  344 
and  also  revealed  that  the  approach  provides  an  important  and  necessary 
technique  for  translating  customer  requirements  for  cost  and  reliability  into  a 
suite  of  goals  and  requirements  that  can  be  measured  and  controlled  in  the 
factory.  In  this  way,  the  HDBK  provides  a  vital  link  for  an  effective  Total 
Quality  Management  (TQM)  program. 
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1.  Introduction 


1.1  Purpose  and  Objectives.  The  purpose  of  DOD-HDBK-344  (HDBK.)  is  to 
describe  the  quantitative  methodology  for  designing,  implementing  and 
monitoring  a  cost  effective  ESS  program.  The  objectives  of  the  study  contract 
were  to  validate  the  concepts  and  procedures  of  the  Handbook  by  applying 
them  to  the  production  of  actual  equipment  in  a  real  world  factory 
environment,  amend  the  Handbook  as  necessary,  and  develop  PC  spreadsheet 
software  to  facilitate  and  automate  the  procedures.  The  study  program  is 
illustrated  in  Figure  1.1.  As  explained  in  this  report,  several  changes  had  to  be 
made  to  the  concept  and  procedures  of  the  Handbook  in  order  to  make  them 
practical  and  viable.  To  overcome  potential  accuracy  problems,  the  procedures 
were  designed  to  maximize  the  use  of  users’  data  collected  and  controlled  in  a 
Statistical  Process  Control  (SPC)  environment.  With  a  general  use  of  these 
procedures  by  the  industry,  many  of  the  existing  uncertainties  in  the 
modelling  equations  and  parameters  will  be  resolved  and  a  more  accurate  data 
base  created. 

The  result  of  the  study  contract  is  a  handbook  on  ESS  methodology  that  has 
been  successfully  applied  in  an  actual  factory  environment.  The  procedures 
developed  are  thus  not  only  believed  to  be  practical  but  provide  a  necessary 
Total  Quality  Management  (TQM)  link  for  translating  customers’ 
requirements  for  cost  and  reliability  into  factory  requirements  that  relate  to 
all  levels  from  design  through  parts  and  materials  procurement,  inspection, 
assembly  and  test. 

1.2  Organization  of  the  report.  This  report  is  organized  in  4  parts.  Part  A 
describes  ESS  concepts  and  methodology,  outlines  the  purpose  and  objectives 
of  the  study  contract,  and  describes  the  type  of  products  designed  and 
manufactured  at  Litton  Systems  (Canada)  Limited  and  in  particular  the 
equipment  used  for  the  study  and  Handbook  validation.  Part  B  describes  the 
application  of  the  HDBK  methodology  to  5  equipments  in  production  at  LSL 
and  the  analysis  techniques  used  tn  compare  observed  and  predicted  results. 
Part  C  describes  the  problems  encountered  with  the  handbook  and  describes 
the  recommended  changes.  Part  D  provides  the  recommended  and  revised 
version  of  DOD-HDBK-344.  The  philosophy  and  mathematical  modelling  for 
ESS  are  discussed  in  Appendix  A.  Appendix  B  contains  a  description  of  the 
software  toolkit  and  discusses  procedures  for  obtaining  copies. 
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PART  A 


L  INTRODUCTION 

This  part  of  the  report  provides  an  introduction  to  the  ESS  concepts  used  in 
DOD-HDBK-344,  and  describes  the  equipment  selected  for  this  study  contract 
as  well  as  an  overview  of  the  other  products  designed  and  manufactured  at 
LSL. 

2.  ESS  Modelling  Principles  and  Concept.  During  the  production  of  electronic 
equipment,  defects  are  introduced  through  flaws  in  parts,  materials, 
workmans?:ip,  and  design  etc.  that  affect  the  production  yields  and  the 
reliability  of  the  equipment.  The  reliability  of  a  given  design  can  be  improved 
by: 

i)  '  reducing  the  number  of  defects  introduced  when  the  equipment  is 
manufactured 

or  ii)  removing  the  defects  prior  to  shipment  through  test,  inspection,and 
the  use  of  ESS. 

The  former  is  the  more  desirable  approach  and  will  usually  lead  to  both 
reduced  production  cost  and  improved  field  reliability.  ESS  is  applicable  to 
both  approaches  and  has  the  principal  purpose  of  exposing  the  existence  and 
cause  of  defects  in  the  equipment  during  development  and  production  factory 
testing.  Provided  the  defect  causes  can  be  eliminated,  ESS  serves  as  an  in 
process  monitoring  tool.  However,  if  the  defect  causes  can  not  be  prevented, 
.then  ESS  serves  the  additional  purpose  of  reducing  the  average  number  of 
defects  per  system  to  a  level  consistent  with  the  reliability  requirements.  . 

In  general,  defects  rates  can  be  modeled  as  the  result  of  eithar  wear  out 
mechanisms  that  affect  the  entire  population,  or  flaws  that  affect  only  a 
portion  of  the  population.  Proper  design  practices,  for  example  integrity 
analyses  and  concurrent  engineering,  should  be  used  to  prevent  design  errors 
and  wear  out  mechanisms.  The  ESS  methodology  should  be  used  to  reduce  the 
defects  and  failure  mechanisms  that  affect  only  a  portion  of  the  population.  In 
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practice,  the  majority  of  parts  used  in  equipment  are  sufficiently  free  of  defects 
so  that,  provided  they  are  properly  used,  reliability  is  not  impacted  during  the 
useful  life  of  the  equipment.  There  is  however  a  small  fraction  of  the  parts  that' 
contain  flaws  that  eventually  cause  equipment  failure.These  flaws  or  defects 
are  the  primary  cause  of  equipment  failure  during  the  early  life  period.  It  is 
this  small  defect  fraction,  usually  measured  in  defects  per  million  (DPM)  that 
is  modeled  and  controlled  by  the  ESS  methodology.  Because  reliability 
concerns  the  fleet  of  equipment,  the  defecte  are  usually  measured  as  defects 
per  unit.  Throughout  this  report,  the  term  defects  thus  refers  to  the  average 
defects  per  system. 

Conceptually,  the  application  of  stress  causes  weakness  or  flaws  in  a  small 
fraction  of  parts  in  the  equipment  to  grow  until  they  cause  equipment  failure. 
The  process  of  transforming  a  flaw  with  some  residual  strength ‘^i,e.^  a  latent 
defect  into  a  defect  capable  of  causing  equipment  failure  i^a  patent  defect  is 
called  defect  precipitation.  The  process  of  detecting  a  patent  defect  is  called 
defect  detection  and  requires  the  proper  test  to  be  performed.  The  stresses  used 
in  ESS  to  precipitate  a  defect  are  typically  combinations  of  temperature, 
vibration,  electrical,  and  occasionally  humidity.  These  stresses  are  used 
because  they  accelerate  defect  precipitation  so  that  the  defect  can  be  exposed 
and  eliminated  by  factory  ESS.  Since  the  factory  stresses  are  representative  of 
field  stresses,  except  at  a  higher  and  thus  accelerating  stress  level,  the  defects 
removed  through  factory  ESS  are  representative  of  the  defects  that  would  have 
otherwise  occurred  in  the  field.  There  is  thus  a  direct  relationship  between  the 
level  of  defects  detected  using  factory  ESS  and  field  reliability.  The  HDBK 
ESS  methodology  thus  addresses  field  reliability  by  quantitatively  modelling 
and  monitoring  defect  levels  both  in  the  factory  and  the  field.  Since  the 
stresses  used  for  ESS  do  not  represent  all  of  the  field  stresses  and  not  all  defect 
mechanisms  can  be  sufficiently  accelerated,  a  factory  ESS  program  can  not  be 
expected  to  be  completely  effective  and  thus  must  be  controlled  using 
statistical  monitoring  techniques; eg.  Statistical  Process  Control  (SPC^ 
supplemented  with  root  cause  physics  of  failure. 

An  Environmental  Stress  Screening  ESS  program,  as  defined  in  MIL-HDBK- 
344,  (HDBK)  provides  quantitative  techniques  for  i)  assessing  the  initial 
number  of  defects,  ie.  defect  density  ii)  designing  an  environmental  stress 
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screening  program  to  remove  and  thereby  reduce  the  defects  to  an  acceptable 
level,  >\s  determined  by  equipment  reliability  requirements,  at  an  optimum 
production  cost,  iii)  determining  and  eliminating  the  causes  of  failures 
through  analysis  and  corrective  action,  and  iv)  applying  statistical  process 
control  techniques  to  monitor  and  control  the  ESS  program. 

ESS  modelling  is  based  on  the  practical  and  mathematical  premise  that  latent 
defects  can  not  be  completely  eliminated  and  that  achieving  zero  defects 
although  an  accepted  ideological  goal  is  unattainable  in  practice.  Some  level 
of  failures  or  failure  rate,  although  being  undesirable,  must  be  expected  and 
thus  be  acceptable.  The  quantitaitive  ESS  methodology  is  concerned  with 
determining  and  achieving  the  level  of  defects  that  can  be  tolerated. 

In  order  to  estimate  the  defects  built  into  the  equipment  and  removed  by  ESS, 
the  methodology  uses  a  combination  of  industry  data  for  defect  levels  and 
screening  strengths,  and  actual  factory  and  field  data  obtained  by  the  ESS 
user.  The  industry  data  contained  in  the  HDBK  is  used  for  initial  planning 
purposes  until  actual  data  is  available.  As  actual  data  becomes  available,  the 
user  analyses  this  data  using  the  methods  described  in  the  handbook  and 
updates  the  ESS  plan  so  that  it  becomes  an  accurate  representation  of  the 
actual  conditions.  In  this  manner,  the  accuracy  of  the  HBBK  methodology  is 
not  limited  by  the  data  contained  in  the  HDBK  that  may  be  generic  and  not 
necessarily  current. 

For  an  effective  ESS  program,  it  is  necessary  to  differentiate  among  the  types 
of  defects.  Defects  are  caused  by  variability  in  parts  and  manufacturing 
processes  and  vary  i)  lot  to  lot,  ii)  with  applied  stress  level  and  type,  and  iii) 
with  time  (life). 

Defects  can  be  characterized  as  latent  or  patent.  A  latent  defect  is  an  inherent 
weakness  or  flaw  with  some  residual  strength  and  will  eventually  cause 
equipment  fdilure  after  exposure  to  s  sufficiently  high  stress  over  some  period 
of  time.  Latent  defects  do  not  cause  a  failure  while  in  the  latent  stage  and  thus 
can  not  be  detected  until  precipitated  as  a  patent  defect.  A  patent  defect  is  a 
defect  that  has  no  residual  strength  and  is  capable  of  causing  an  immediate 
equipment  failure  given  the  proper  circumstances.  It  can  therefore  be  detected 
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in  its  present  state  provided  the  proper  test  is  performed.  Patent  defects  can  be 
caused  by  a  precipitated  latent  defect  or  manufacturing  "error".  The 
precipitation  of  latent  defects  into  patent  defects  determines  the  equipment’s 
reliability.  Errors,  such  as  test  correlation,  missing  or  ipcorrect  parts  etc  are 
usually  preventable  or  readily  detectable  and  are  typically  removed  from  the 
hardware  before  shipment  provided  proper  factory  tests  are  performed. 
Although  errors  have  the  potential  for  impacting  reliability,  they  can  be 
eliminated  without  ESS  and  tend  to  impact  production  cost  more  than  field 
reliability.  The  causes  of  errors  are  different  than  latent  defects  apd  thus 
should  be  addressed  separately  from  latent  defects  foi  ESS  planning  and 
monitoring. 

The  ESS  methodology  fulfills  a  vital  and  necessary  prerequisite  for  an  effective 
Total  Quality  Management  (TQM)  program  since  it  provides  the  technique  for 
translating  a  customer's  requirements  for  cost  and  reliability  into  a  suite  of 
design  and  factory  requirements  that  are  thus  directly  related  to  the 
customer's  perception  of  satisfaction.  The  tools  provide  management  with 
early  visibility  on  the  degree  to  which  these  goals  are  being  achieved  and  an 
analysis  capability  to  identify  and  resolve  problems  and  to  optimize  the 
allocation  of  resources.  The  methodology  thus  ensures  that  the  customer 
receives  equipment  that  meets  reliability  requirements  with  an  optimum 
screening  cost. 

3.  Litton  Systems  Canada  Limited.  Litton  Systems  Canada  Limited  (LSL)  is 
a  major  operating  division  of  Litton  Industries,  and  has  been  designing  and 
manufacturing  sophisticated  electronics  equipment  for  military  and 
commercial  use  for  over  25  years.  Although  initially  involved  in  Inertial 
Navigation  Systems  (INS),  LSL  has  expanded  the  manufacturing,  design,  and 
system  engineering  capabilities  to  a  diversified  capability  and  range  of 
products. 

3.1  Inertial  Navigation  Systems.  Litton  systems  Canada  Limited  (LSL)  was 
established  in  1960  as  an  inertial  navigation  systems  house.  Having 
demonstrated  the  capability  to  manufacture  INS,  LSL  has  continued  to 
expand  and  develop  its  INS- related  capabilities  to  cover  a  range  of  both 
military  and  civil  systems.  This  has  resulted  in  a  facility  with  total  capability 
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in  manufacturing,  engineering  and  systems  management.  The  production 
facility  meets  the  critical  standards  of  the  military  and  civil  aviation 
org^izations. 

To  date,  more  than  12,000  inertial  navigation  systems  have  been  produced  at 
LSL,  representing  a  significant  portion  of  all  commercial  inertial  navigation 
systems  in  use  worldwide.  Military  systems  manufactured  by  LSL  are  the 
LN-3-2A  and  2B  for  the  F-104,  LN-12  for  F-4,  LN-14  for  the  F-111,  LN-33L  for 
CP-140,  LW-33CF  for  the  CF-104  retrofit,  and  Equipment  A  which  is  being 
produced  by  LSL  for  the  U.S.  Cruise  Missile  on  a  second  source  basis. 

Production  of  Equipment  A  has  led  to  the  introduction  of  even  higher 
standards  of  manufacturing  facilities.  Commercial  inertial  systems 
manufactured  by  LSL  are  the  LTN-51,  LTN-58,  LTN-70  series  and  Equipment 
C  series,  used  by  airlines,  in  civil  and  military  transport  applications  and  in 
business  aircraft.  The  LTN-76  is  an  INS  used  in  special  applications  where 
extra  precision  is  required. 

LSL’s  inertial  systems  provide  reliable,  self-contained,  all  weather,  worldwide 
navigation  capability  that  is  independent  of  external  navigation  aids.  They 
are  primarily  designed  for  long-range  aircraft,  particularly  on  overseas  routes, 
and  they  supply  continuous,  accurate  position,  navigation  and  guidance  data. 
In  addition,  they  are  often  used  as  the  primary  source  of  aircraft  altitude  and 
heading  information. 

3.2  Flight  Inspection  Systems.  The  Inertial  Referenced  Flight  Inspection 
System  (IRFIS)  was  developed  by  Litton  Systems  Canada  Limited  as  a  result  of 
the  desire  of  the  Canadian  Ministry  of  Transport  to  automate  their  Instrument 
Landing  System  (ILS)  calibration  facilities.  Traditionally,  Transport  Canada, 
like  many  worldwide  civil  and  military  aviation  agencies,  calibrated  ILS  using 
manually  operated  theodolite  techniques.  Following  an  extensive  series  of 
studies  and  simulations  performed  by  LSL,  the  Ministry  awarded  LSL  a 
contract  to  design  and  build  a  more  accurate,  real-time  calibration  system,  the 
IRFIS.  The  system  underwent  successful  flight  trials  during  the  Spring  of 
1978  at  the  Canadian  Air  Force  (CAF)  photo-theodolite  range  at  Cold  Lake, 
Alberta,  and  has  proven  capability  to  calibrate  Category  1, 11  and  n  ILS  as  well 
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as  en  route  navaids.  The  IRFIS  represents  the  most  sophisticated  and 
advanced  system  of  its  type  currently  available. 

3.3  Automatic  Test  Equipment  Capability.  LSL  has  been  active  in  the  design 
and  development  of  general  purpose  Automated  Test  Equipment  (ATE)  for 
Military  applications  since  the  mid-1960s.  ATE  and  LSL  designed  and 
developed  Test  Program  Sets  (TPSs)  have  been  supplied  to,  and  are  in  service 
with,  military  forces  around  the  world.  From  the  experience  gained,  a  family 
of  ATE  has  evolved  which  is  capable  of  providing  comprehensive,  cost-effective 
support,  at  the  Intermediate  and  Depot  levels,  for  a  wide  range  of  electronic 
and  hydraulic  systems. 

The  basic  member  of  this  family  is  the  Expanded  Litton  Automated  Test  Set 
(ELATS),  which  performs  test  and  fault  isolation  on  digital,  analog  and  hybrid 
Weapons  Replaceable  Assemblies  (WRAs)  and  Shop  Replaceable  Assemblies 
(SRAs).  For  testing  WRAs  and  SRAs  which  operate  in  the  radio  frequency 
,  range,  LSL  has  developed  the  RF/ELATS. 

LSL  has  produced  over  four  hundred  TPS  for  a  wide  variety  of  electronic  items. 
The  electronic  design,  software  and  technical  documentation  associated  with 
TPSs,  can  be  produced  either  to  best  commercial  standards  or  to  full  Military 
Specifications  as  demanded  by  such  customers  as  the  U.S.  Air  Force  and  the 
U.S.  Navy.  All  Test  Programs  supplied  for  execution  on  the  ELATS  family  are 
written  in  IEEE  716  ATLAS  (Abbreviated  Test  Language  for  All  Systems), 
which  is  an  ARINC  and  U.S.  DOD  Standard. 

3.4  Displays. 

3.4.1  LED  Display  Systems.  Since  June  1975,  LSL,  initially  under  contract 
with  the  USAF  and  the  Canadian  Government  and  later  under  contract  with 
the  Canadian  Government  alone,  has  developed  a  solid-state,  flat-panel 
display  system  using  Light  Emitting  Diode  (LED)  technology  for  use  in 
military  environments.  Throughout  the  program  the  Flight  Dynamics 
Laboratory  of  the  USAF  at  Wright  -Patterson  Air  Force  Base  acted  as  the 
design  authority. 
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LSL  realized  initial  success  with  General  Dynamics  for  Data  Entry  Display 
(DED)  systems  on  the  F-16C  and  F-16D  Fighting  Falcon  aircraft.  The  full- 
scale  development  phase  was  completed  and  the  first  of  over  2,000  production 
DEDs  was  delivered  to  General  Dynamics  in  September,  1983.  Strong  growth 
in  this  contract  and  follow-on  contracts  continues  today.  Application  of  the 
base  technology  developed  for  LED  display  products  lead  to  the  design  of 
programmable  multifunction  keypads.  This  prodifct  is  used  in  strategic 
system  control  consoles  in  EC- 135  aircraft. 

3.4.2  Programmable  Display  Module.  During  the  course  of  1984,  LSL  adopted 
a  vertically  integrated  approach  to  its  display  systems  capability.  While  the 
company’s  major  thrust  remains  directed  at  the  large-scale  complete  systems 
opportunities,  LSL  nevertheless  has  expanded  its  efforts  in  the  development  of 
a  modular  or  component  capability. 

The  PDM  has  been  designed  specifically  to  meet  the  display  requirements  for  a 
programmable  switch  or  annunciator.  Each  of  the  PDMs  560  Light  Emitting 
Diodes  (LEDs),  configured  in  a  16-inch  by  35-inch  array,  can  be  individually 
addressed  providing  a  complete  alphanumeric  and  graphic  capability. 

Configured  in  an  annunciator  panel,  the  PDM  effectively  saves  valuable 
cockpit  or  console  space,  reduces  weight  and  provides  a  more  cost-effective 
alternative  to  panels  currently  in  use.  Through  combining  several  functions 
within  a  single  PDM,  the  design  engineer  can  create  an  annunciator  panel 
which,  in  addition  to  the  above  advantages,  is  also  beneficial  from  a  human 
factors  standpoint.  Further,  at  point  of  retrofit,  it  does  not  become  necessary  to 
replace  the  panel,  merely  reprogram  the  PDMs. 

3.4.3  Liquid  Crystal  Display.  To  supp^ment  the  flat  panel  display  product 
line,  LSL  has  designed  and  is  expanding  a  manufacturing  capability  for  Thin 
Film  Transistor  (TFT)  active  matrix  liquid  crystal  displays.  Colour  displays 
up  to  6  X  8  inch  dimensions  with  80  (colour)  pixels/inch  as  well  as  monochrome 
displays  have  been  produced. 

3.4.4  Advanced  Cockpit  Display  Systems.  Starting  in  1984  LSL  initiated  a 
major  research  and  development  program  to  establish  display  system 
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technologies  appropriate  to  the  cockpit  display  requirements  of  advanced 
military  and  commercial  aircraft,  scheduled  for  production  in  the  mid-1990s. 
Evolving  from  this  program  are  three  distinct  product  types  namely;  colour 
active  matrix  liquid  crystal  displays,  graphic  processor  modules  and  SrD 
graphics  software. 

LSL’s  colour  active  matrix  display  technology  offers  a  low-cost,  lightweight 
reliable  alternative  to  the  multifunction  shadow  mask  colour  cathode  ray  tube 
display.  ,  ' 

LSL’s  Graphic  Processor  Module  (GPM)  is  being  designed  in  cognizance  of  the 
United  States  Department  of  Defense  Joint  Integrated  Avionics  Plan  for  new 
aircraft.  LSL’s  GPM  will  enable  the  display  processing  function  to  be 
implemented  within  an  advanced  avionic  architecture  based  on  the  common 
module  concept.  The  display  processing  function  is  to  generate  real-time 
complex  3-D  imagery  consistent  with  advanced  display  formats  intended  to 
increase  pilot  situational  awareness  and  to  decrease  pilot  workload. 

LSL’s  3-D  graphic  software,  referred  to  as  the  Litton  Graphics  System  (LGS),  is 
a  set  of  computer  graphics  interface  commands,  derived  from  the 
Programmer’s  Hierarchical  Interface  Graphic  Standard  (PHIGS)  functionality 
for  controlling  the  definition,  display  and  modification  of  graphics  data.  LGS 
exists  in  an  ADA  language  and  is  transportable  from  the  application 
development  workstation  environment  to  the  advanced  avionic  system 
environment. 

3i5  Airborne  Radar  Systems.  LSL  entered  the  airborne  search  radar  field  in 
1972  when  the  AN/APS-503  Airborne  Radar  System  was  designed  and 
developed. 

Subsequently,  the  APS-504  (V)2,  a  100kW,(peak)  X-band  airborne  search 
radar  system  was  designed  for  maritime  search  application,  particularly 
against  small  targets  in  high  sea-states.  Functions  include  maritime 
surveillance,  station  keeping  and  mapping  with  a  detection  range  extended 
out  to  250  nautical  miles.  LSL  (V)2  systems  are  now  installed  in  seven  types  of 
aircraft,  currently  operational  in  seventeen  countries  around  the  world. 


Continuing  development  of  this  family  of  radar  produced  the  APS-504(V)5 
[AN/APS-140(V)].  This  latest  generation  radar  provides  a  sophisticated 
Maritime  Patrol  and  ASW  capability.  While  retaining  the  Held-proven  scan 
converter  and  TV  raster  display  of  the  (V)3,  the  (V)5  adds  wideband  frequency 
agility,  500:1  pulse  compression  using  surface  Acoustic  Wave  (SAW) 
technology,  extensive  sea  clutter  rejection  circuitry  and  a  Track- While-Scan 
(TWS)  mode.  Further  development  of  the  (V)5  to  include  Synthetic  Aperture 
and  Pulse  Doppler  techniques  is  currently  being  implemented. 

3.5.1  Systems  Engineering.  Since  1966,  when  Litton  Systems  Canada 
Limited  was  placed  under  contract  to  provide  the  CCS-280  system  for  the 
Canadian  navy’s  DDH-280  class  destroyers,  LSL  has  been  the  largest  supplier 
in  Canada  of  sophisticated  Command,  Control,  Communications  and 
Intelligence  (C3I)  information  processing  systems. 

LSL  has  since  expanded  its  scope  to  related  roles,  including  Sustaining 
Engineering  for  the  CP- 140  Aurora  Anti-Submarine  Warfare  (ASW)/Maritime 
Patrol  Aircraft  (MPA)  Avioaics/Electron*cs,  Weapon  System  Integration,  such 
as  Command,  Control  and  Communications  Engineering  for  the  Canadian 
Forces  Low  Level  Air  Defence  System  (LLADS)  and  also.  Prime  Contractor 
responsibility  for  the  Tribal  Class  Update  and  Modernization  Project 
(TRUMP). 

3.6  Naval  Systems  Engineering. 

3.6.1  Tribal  Class  Update  and  Modernization  Project  (TRUMP).  Litton 
Systems  Canada  Limited  was  awarded  the  contract  for  the  Tribal  Class 
Update  and  Modernization  Project  (TRUMP)  in  1986  to  modernize  and  equip 
the  ships  with  area  air  defence  systems  to  meet  the  threat  of  the  1990s. 

3.6.2  Automatic  Data  Link  Information  system.  The  Automatic  Data  Link 
Information  Plotting  System  (ADLIPS)  is  a  low-cost,  shipborne,  computer- 
assisted,  real-time  command,  control  and  tactical  data  communications  system 
which  can  be  fully  integrated  with  existing  ship  systems.  The  system  collects 
and  processes  on-ship  sensor  data  to  generate  a  composite  tactical  picture  to 


11 


facilitate  operator  action.  It  enables  the  ADLIPS  fitted  ships  to  communicate 
with  each  other,  and  with  other  ships  and  aircraft  which  already  have  data 
link  capability,  permitting  the  performance  of  coordinated  tactical  operations. 

3.7  Airborne  Systems  Engineering. 

3.7.1  CP- 140  Sustaining  Engineering.  Involvement  in  the  CP-140  program 
began  in  the  mid-70s  when  LSL  participated  in  the  contract  definition  phase  of 
the  aircraft  procurement  program.  Following  contract  award  to  Lockheed, 
LSL  was  selected  as  a  subcontractor  to  Lockheed  to  develop  and  install  the  two 
DIACs,  and  to  develop  and  produce  the  dual  LN-33L  Inertial  Navigation 
System  installed  in  each  aircraft:  The  Litton  Automated  Test  Set  (LATS)  was 
also  provided  for  ground  maintenance  support  of  the  LN-33L  and  other 
avionics  systems. 

In  1982,  LSL  was  selected  by  the  Canadian  Defence  Department  to  provide 
third- level  maintenance  for  the  CP- 140  avionics  systems. 

3.7.2  Navigation/Tactical  Computer  and  Display  (NAV/TAC)  System.  The 
NAV/TAC  is  a  low-cost,  militarized,  centrally  controlled  tactical  information 
and  display  processing  system.  The  system  is  based  on  the  Motorola  68020 
microprocessor  and  provides  sensor  control  and  mission  control  for  an  Anti¬ 
submarine  Warfare  (ASW)  mission  avionics  suite.  The  system  is  designed  for 
smaller  fixed-wing  or  rotary-wing  aircraft.  The  basic  system  is  comprised  of 
four  major  subsystems 

a)  A  Tactical  processor 

b)  An  Input/Output  processor 

c)  A  Display  processor 

d)  A  Map  processor. 

Each  subsystem  includes  an  independent  32-bit  microprocessor  with  local 
memory.  In  addition  special  processors  are  used  for  floating  point  calculations. 
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vector  graphic  generation,  and  map  data  expansion.  The  system  architecture 
is  based  on  the  VME-bus  for  system  interconnect,  thus  allowing  a  full  32-bit 
data  path  between  subsystems.  The  display  processor  provides  both  vector  and 
pattern  graphics  on  a  high-resolution  bitmapped  RGB  colour  video  display. 
Maps  of  1,  024  nautical  miles  square  can  be  generated,  stored,  and  displayed 
with  a  resolution  of  100  years  with  four  planes  of  video  bit  map  (1,280  by  1,024 
by  4  bits). 

3.7,3  Helicopter  Integrated  Processing  and  Display  System  (HINPADS).  LSL 
is  teamed  with  Computing  Devices  Company  and  Canadian  Marconi  Company 
to  design  and  develop  the  Helicopter  Integrated  Processing  and  Display 
System  (HINPADS)  Advanced  Development  Module  (ADM).  HINPADS  has 
been  proposed  as  the  core  avionics  suite  for  the  New  Shipbome  Aircraft  (NSA) 
program  to  satisfy  the  requii^ements  for  a  Mission  Systems  Data  Handling 
Subsystem  (MSDHS),  arou'^d  which  the  NSA  mission  avionics  can  be 
integrated.  The  HINPADS  ADM  will  provide  the  Canadian  Department  of 
National  Defence  (DND)  with  a  validated,  developmental  model  of  a  system 
which  will  provide  integrated  processing,  control  and  display  functions  for  a 
distributed,  mission-oriented  avionics' suite. 

3.8  Ground-Based  Systems  Engineering. 

3.8.1  Data  Interpretation  and  Analysis  Centre  (DIAC).  In  1976,  LSL  was 
placed  under  contract  by  Lockheed  California  Company  to  provide  a 
computerized,  ground-based,  data  processing  system  to  support  the  CP- 140 
Aurora  ASW.MPA  aircraft  for  the  Canadian  Forces.  By  1981,  LSL  had 
delivered  and  installed  two  DIAC  systems,  at  Greenwood,  Nova  Scotia  and 
Conjox,,  British  Columbia.  These  systems  provide  the  Canadian  Forces  with 
aids  for  the  briefing  and  debriefing  of  aircrews,  mission  planning,  mission 
reconstruction  and  replay,  the  maintenance  in  dissemination  of  the  tactical 
situation  for  ASW,  SAR,  fisheries  protection  and  sovereignty  patrol  missions. 

3.8.2  Canadian  Forces  Low-Level  Air  Defence  System  (CF  LLADS).  LSL  is 
playing  a  major  role  in  the  CF  LLAD  program.  In  support  of  Oerlikon 
Aerospace  (OA)  of  St.  Jean,  Quebec,  LSL  is  providing  essential  production  of 
major  electronics  subsystems  as  well  as  playing  a  leading  roie  in  systems 
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engineering  by  creating,  developing,  and  supplying  a  sophisticated,  high 
performance  C3  system. 

There  are  two  major  electronic  production  projects  for  LSL,  each  of  which  hes 
required  a  significant  technology  transfer  and  facility  change.  The  production 
of  Turret  Electronics  assemblies  and  electro-optical  control  consoles  was 
preceded  by  extensive  LSL  participation  in  two  separate  subprojects.  LSL 
undertook  the  maturation  of  the  electronics  design  in  order  to  facilitate 
production  and  to  enhance  reliability.  Also,  LSL  hosted  the  OA  and  Martin 
Marietta-Oerlikon  Aerospace  (MMOA)  teams  for  the  Pathfinder  program 
which  integrated  a  complete  weapons  system  using  the  production  level 
electronics. 

3.8.3  Security  Systems.  Litton  Integrated  Security  Systems  arc  turnkey 
installations  which  rely  on  LSL’s  extensive  systems  design  capabilities. 
Typically,  a  group  of  subsystems  providing  functions  such  as  perimeter  and 
interior  intrusion  detection,  entry  control,  CCTV  surveillance,  radio 
communicati  jns  cmd  automatic  fault  detection  are  coordinated  and  controlled 
by  dual  redundant  computers. 

Litton  Systems  Canada  Limited  has  developed  three  integrated  command  and 
control  systems  for  Correctional  Services  of  Canada  and  has  installed  them  in 
maximum  security  penal  institutions  at  Saskatoon,  Saskatchewan;  Edmonton, 
Alberta  and  Agassiz,  British  Columbia.  The  systems  use  an  acoustic  fence 
sensor  combined  with  a  sterile  zone  monitored  by  microwave  signals  to  detect 
attempted  penetration  of  the  perimeter  fences. 

LSL  has  supplied  several  perimeter  and  access  monitoring  Systems  tc  Ontario 
Hydro  for  protection  of  nuclear  power  generating  stations.  The  systems 
installed  provide  two  separate  and  complementary  methods  of  detecting 
intrusions  into  restricted  areas  so  that  a  temporary  single-point  failure  will 
not  degrade  the  level  of  detection. 

LSL’s  most  recent  success  in  security  systems  was  realized  when  a  competitive 
bid  was  won  to  install  a  major  security  system  for  the  Royal  Saudi  Air  Force 
(RSAF)  base  at  Dhahran,  Saudi  Arabia. 
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3.9  Systems  Engineering  Support  -  Classified  Modelling  Facility.  In  support 
of  its  system  engineering  operations,  and  in  support  of  the  Canadian 
Governnient’s  Defence  Researc  ■  needs,  LSL  operates  a  Classified  Modelling 
Facility.  This  facility  is  available  to  both  industry  (including  LSL  itselD  and  to 
government,  on  a  contract  basis.  The  facility  is  TEMPEST-secure,  and  is 
certified  for  operations  to  the  SECRET  level.  Within  the  TEMPEST  enclosure, 
LSL  operates  a  Digital  Equipment  Corporation  (DEC)  VAX-11/750  computer 
and  a  DEC  2020  computer. 

The  configuration  of  these  systems  is  compatible  with  the  equivalent 
processors  at  the  Defence  Research  Establishment,  Ottawa,  (DREO),  and  the 
Operational  Research  and  Analysis  Establishment  (AE),  to  allow  models  to  be 
transferred  to  the  Government  so  that  they  may  exercise  without  Contractor 
involvement.  The  CMF  has  available  a  number  of  defence  scientists  and 
software  staff,  capable  of  supporting  applications  in  radar,  electronic  warfare 
(EW),  communications,  operations  research  and  systems  effectiveness 
analysis. 

4.  Description  of  the  equipment  used  in  the  evaluation.  As  required  in  the 
RADC  statement  of  work,  LSL  selected  three  different  types  of  equipment  for 
the  study  contract.  In  order  to  have  an  adequate  data  base  for  analysis 
purposes,  equipment,  with  2  or  more  years  of  production  history  were  selected. 
Although  LSL  both  designs  and  manufactures  complex  electronic  equipment, 
LSL  also  manufactures  equipment  to  other  companies  designs  and  drawing 
packages.  The  types  of  equipment  selected  for  the  study  were  "build  to  print" 
programs  designed  by  three  different  companies.  The  selection  of  "build  to 
print"  over  LSL  designed  equipment  was  made  in  the  interest  of  greater 
objectivity  in  the  data  analysis,  and  also  to  provide  a  greater  cross  section  of 
companies  design  approaches  and  part  types.  In  this  way  the  study  was  made 
more  representative  of  the  military  electronics  industry  in  general.  The 
selection  was  also  made  to  include  a  wider  range  of  part  reliability  grades  and 
packaging  methods.  Equipment  A  for  example  uses  high  reliability  parts  that 
are  further  upgraded  through  100%  parts  rescreening  at  LSL.  Equipment  A 
uses  both  flat  pack  and  through  hole  packaging  and  thus  contains  hand  and 
wave  soldered  connections.  Equipment  B  use  high  reliability  grade  parts  with 
through  hole  packaging  and  thus  predominantly  wave  soldered  connections. 


Equipment  C  uses  high  reliability  grade  parts  but  makes  more  extensive  use  of 
MILrSTD-883  level  . as  opposed  to  full  MIL-M-38510  level  devices  and  is 
assembled  using  through  hole  and  wave  soldered  connections. 

The  i*eliability  and  ESS  programs  applicable  to  the  selected  equipment  are 
provided  in  the  following.  Details  on  the  equipment  complexity  can  be  found  in 
Part  B. 

4.1  ESS  Programs  used  on  the  selected  equipment 

4.1.1  Equipment  A.  Military  Inertial  Navigation  System.  This  equipment 
provides  the  inertial  reference  measuring  unit  and  computer  for  a  missile 
guidance  set.  The  equipment  consists  of  23  complex  electronic  and/or 
electromechanical  assemblies  and  contains  approximately  3000  electronic 
parts.  ESS  is  presently  used  at  all  levels  as  follows: 

i)  100%  parts  screening  and  testing  at  vendor 

ii )  100%  parts  testing  and  rescreening  at  LSL  Receiving  Inspection 

iii)  Assembly  (card)  level  ESS  and  testing  using  automatic  test 
equipment  (ATE)  (e.g.  in  circuit  testers)  and/or  performance  and 
functional  testing 

iv)  system  level  ESS  and  testing  using  ATE 

v)  system  level  failure  free  testing  and  reliability  verification  testing 
(PRVT) 

The  ESS  flow  is  shown  in  Figure  4.1. 


16 


®  NOT  INCLUDED  IN  THIS  STUDY 


FIGURE  4.1  EQUIPMENT  A  ESS  Flow. 


4.1.2  Eouipment  B,  Display  Svstbm  For  Fighter  Aircraft.  This  equipment 
provides  the  displays  for  a  fighter  aircraft  and  consists  of  Heads  Up  Display 
(HUD),  Multipurpose  Display  Indicator  (MDI),  and  Multipurpose  Display 
Indicator  Repeater  (MDRI).  Each  HUD  consists  of  18  complex  electronic 
assemblies,  each  MDI  consists  of  22  complex  electronic  assemblies  and  each 
MDRI  consists  of  8  complex  electronic  assemblies. 

The  ESS  program  consists  of  part  screening  and  testing  at  receiving 
inspection,  assembly  (card)  level  testing,  and,  system  level  ESS.  The  outline  of 
the  ESS  flow  is  shown  in  Figure  4.2. 

4.1.3  Equipment  C.  Inertial  Navigation  System  For  (Commercial  Aircraft. 
Equipment  C  provides  inertial  navigation  and  reference  systems  for 
commercial  and/or  military  transport  aircraft.  These  are  advanced  systems 
using  ring  laser  gyroscopes  and,  because  of  the  application  have  quality  and 
reliability  programs  and  requirements  similar  to  military  equipment.  For 
example,  parts  used  are  Quality  Grade  3  as  defined  in  MIL-HDBK-344.  The 
factory  testing  and  ESS  program  includes  part  testing  and  rescreening  at 
receiving  inspection  on  a  sample  and/or  100%  basis,  assembly  level  ESS  and 
testing,  and  system  level  ESS  and  testing.  The  ESS  test  flow  is  provided  in 
Figure  4.3. 

4.2  Reliability  Program  Requirements  for  the  selected  equipment.  The  three 
equipment  types  selected  for  the  ESS  study  have  quality  programs  conducted 
in  accordance  with  MIL-Q-9858.  Equipment  A  &  B  have  customer  specified 
reliability  programs  that  adhere  to  the  tasks  as  specified  in  MIL-STD-785. 
The  program  for  Equipment  C  has  requirements  which  are  similar  or 
equivalent  in  purpose  as  noted  in  Table  4.1  through  4.3. 

Tables  4.1  and  4.2  list  the  MIL-STD-785  tasks  that  are  required  by 
specification  and/or  imposed  by  LSL  procedures  and  manuals,  and  list  the 
tasks  supported  by  the  program  plan.  Table  4.3  lists  the  ESS  related 
specifications  that  are  imposed  by  the  customers. 
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<j)  MDRI  and  HUD/MDI  respectively.  All  systems  receive  40  Temp 
Cycles  in  total. 


FIGURE  4.3  Equipment  B  ESS  Flow. 
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FIGURE  4.3  Equipment  C  ESS  Flow. 
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TABLE  4.1  Equipment  Program  Surveillance  and  Control  Tasks. 


TASK# 

MIL-STD-785  TASK 

Equip¬ 
ment  A 

Equip¬ 
ment  B 

Equip¬ 
ment  C 

101 

Reliability  Program  Plan 

(1) 

(2) 

102 

Monitor/Control 

Subcontractors 

X 

X 

103 

Program  Reviews 

X 

X 

104 

Failure  Reporting, 

Analysis  and  Corrective 
Action 

X 

X 

(3) 

105  - 

Failure  Review  Bosird 

X 

X 

(3) 

NOTES; 


1.  Program  to  MIL-STD-785  Rev.  B. 

2.  Program  to  MIL-STD-785  Rev.  A. 

3.  Commercial  equivalent,  integrated  with  LSL’s  FRACAS  system  which 
uses  the  same  failure  remrting,  anal^s  and  corrective  action  action 
elements  that  are  shared  with  the  military  programs 
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TABLE  4.2  Ecuipment  Design  and  Evaluation  Tasks. 


TASK 
.  # 

MIL-STD-785BTASK 

Equip¬ 
ment  A 

Equip¬ 

ments 

Equip¬ 
ment  C 

201 

Reliability  Model 

X 

X 

(1) 

202 

Reliability  Allocations 

X 

X 

203 

Reliability  Predictions 

X 

X 

(1) 

204 

FMECA 

X 

X 

(1) 

205 

Sneak  Circuit  Analysis 

(2) 

207 

Parts  Program 

X 

X 

(1) 

209 

Effects  of  Testing  Storage  and 
Transportation 

X 

(3) 

301 

Environmental  Screening 
[ESS] 

X 

X 

X 

302 

Rel  Dev.  Growth  Test  [RDGT] 

(4) 

X 

(5) 

303 

Rel  Qualification  Test  [RQT] 

(4) 

X 

304 

Production  Rel  Acceptance 

Test  [PRAT] 

X 

X 

(6) 

NOTES; 

1.  Completed  to  commercial  equivalent  to  military  requirements.  The 
main  difference  is  no  formal  data  item  description  [DID]. 

2.  Informal  (no  DID)  sneak  circuit  analysis  required.  Results  available 
upon  request  by  customer. 

3.  Only  description  of  packaging  methods  and  materials  required. 

4.  Verification  by  analysis,  supplemented  by  field  data. 

5.  Completed  to  commercial  equivalent  of  the  military  requirements.  No 
formal  data  items  descriptions.  Reports  tailored  to  company  standards. 

6.  Commercial  equivalent,  integratea  with  LSL’s  FRACAS  system  which 
shares  the  same  failure  reporting,  analysis  and  corrective  action 
elements  with  tiie  other  in-house  military  pro^ams. 
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TABLE  4^  Miscellaneoi^  ’ESS  Related  Specifications’  for  the 

Equipments. 


SPECIFICATION 

ITEMDESCRIPnON 

Equip¬ 
ment  A 

Equip¬ 
ment  B 

Equip¬ 
ment  C 

MIL-STD-965 

Parts  Control  Program 

X 

X 

(1) 

MIL-STD-1520 

Corrective  Action  System 

X 

X 

X 

NAVMATP9492 

Navy  Manufacturing 
Screening  Program 

X 

(1) 

(2) 

MIL-STD-781 

Reliability  Tests, 
Production 

X 

X 

MIL-STD*1631A 

Procedure  for  Selection  of 
Electronic  Parts 

X 

MIL-STD-1535 

Supplier  Quality 

Assurance  Program 
Requirements 

X 

MIL-STD-1695 

Mil  Standards  for  Working 
Environment 

X 

NAVMATINST 

Contractual 

X 

4855.10 

Manufacturing 

Requirements 

MIL-Q-9858 

Quality  Program 
Requirements 

X 

X 

X 

1.  Customer  specified  equivalent. 

2.  Litton  specified  equivalent. 
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PARTS 

1.  Introduction.  The  procedures  of  DOD-HDBK-344  were  applied  to  the 
equipment  described  in  Part  A  in  order  to  assess  the  methodology  and 
handbook  accuracy  under  factory  conditions.  This  part  of  the  report  describes 
the  application  of  the  handbook’s  procedures,  the  methods  used  to  collect  and 
analyze  factory  data,  and  conclusions  and  recommendations  from  the  analyses. 

2.  Application  of  DOD-HDBK-344 

2.1  ESS  Application  Constraints.  The  ESS  procedures  for  designing  and 
monitoring  an  ESS  program  are  described  in  Section  5  of  the  handbook. 
Because  the  equipment  selected  by  LSL  was  currently  in  production,  it  was 
necessary  to  apply  the  HDBK  procedures  in  retrospect.  Further,  the  ESS 
performed  on  the  equipment  was  defined  contractually  and  thus  could  not  be 
altered  to  suit  the  studj'  contract.  The  study  program  approach  was  thus  to  use 
the  HDBK  to  model  the  actual  ESS  being  conducted  on  the  three  equipment 
types  and  then  to  compare  the  observed  factory  results  to  the  HDBK’s 
predictions.  The  viability  of  the  methodology  and  procedures  and  the  HDBK 
accuracy  could  thus  be  assessed.  The  analysis  techniques  used  and  the  results 
are  discussed  in  this  section. 

2.2.  DOD-HD3K-344  Methodology  Overview. 


2.2.1  Methodology  Overview.  The  ESS  concept  and  methodology  described  in 
the  HDBK  is  a  quantitative  approach  that  establishes  a  relationship  between 
field  reliability  and  the  defects  remaining  in  equipment  at  the  time  of 
shipment.  The  field  reliability  requirement  can  be  deHned  in  terms  of  an 
average  MTBF  or  failure  free  percentage  measured  over  a  specified  time 
interval  and  conditions.  The  ESS  methodology  directly  relates  these 
requirements  to  a  maximum  level  of  defects  per  system  that  can  exist  when  the 
equipment  completes  factory  testing.  The  methodology  also  estimates  the  level 
of  defects  that  exist  when  the  equipment  is  manufactured.  The  ratio  of  the 
maximum  allowable  defects  per  syctem  at  time  of  shipment  to  the  number  of 
defects  per  system  introduced  during  manufacturing  determines  what  fraction 
of  defects  must  be  removed  through  inspection, testing,  and  ESS.  The 
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effectiveness  of  removing  these  defects  is  quantitatively  measured  by  the 
screening  strength  of  ESS.  Screening  strength  is  governed  by  the  ability  to 
precipitate  a  latent  defect  to  a  state  where  it  can  be  detected,  ie  precipitation 
efficiency,  and  the  the  ability  of  a  test  to  detect  and  isolate  the  precipitated 
defect  ie  detection  efficiency.  The  stresses  used  to  precipitated  and  detect 
defects  are  representative  of  the  stresses  that  the  equipment  will  see  in 
operation,  storage,  transportation,  and  maintenance  etc  but  at  higher  and  tkus 
accelerating  levels.  The  defects  removed  through  factory  ESS  are  thus  similar 
to  the  defects  that  would  have  occurred  in  the  field  if  ESS  had  not  been 
performed  and  thus,  there  is  relationship  between  factory  fallout  and  field 
reliability  for  both  causative  effects  and  levels. 

To  apply  the  ESS  methodology,  the  user  of  the  HDBK  must  determine  the 
following  key  parameters: 

i)  Incoming  defect  levels.  Although  the  majority  of  parts  and 
workmanship  operations  are  defect  free,  some  small  fraction  will 
contain  flaws  or  defects  that  can  be  measured  as  a  defect  density  in 
defects  per  million.  The  sum  of  the  defects  from  parts  and 
workmanship  operations  determines  the  total  defects  per  system.  It 
is  the  average  defects  per  part,  workmanship  operation,  and  thus 
system  that  must  be  determined  to  apply  the  ESS  methodology. 

ii)  Screening  Strength.  The  probability  of  a  screen  being  able  to 
precipitate,  detect,  and  remove  a  defect  is  measured  by  its  screening 
strength.  Screening  strength  is  the  product  of  precipitation 
efficiency,  determined  from  the  type,  level  and  duration  of  the 
stress,  and  detection  efficiency,  determined  from  the  probability 
that  a  test  will  detect,  and  isolate  a  particular  fault. 

These  parameters  and  other  HDBK  requirements  are  discussed  in  the  sections 
that  follow. 

2.2.2  Estimating  Incoming  Defects.  The  ESS  methodology  as  described  in  the 
HDBK  procedures  estimates  the  incoming  defects  based  upon  the  equipment 
complexity  and  the  reliability  level  of  the  various  parts  and  processes.  This  is 
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accomplished  by  describing  the  equipment  in  terms  of  the  quantity  of  items 
within  various  part-process  categories  and  multiplying  these  by  an  expected 
fraction  defective  for  each  of  the  part-reliability  grades  and  process  types.  The 
fraction  defectives  are  determined  using  lookup  tables  found  in  the  HDBK  and 
have  different  values  depending  upon  the  end  application  operating 
environment.  The  term  fraction  defective  is  used  in  the  handbook  to  describe 
the  number  of  defective  electronic  parts  per  million  used.  For  consistency  ,it  is 
recommended  that  the  term  Defect  Density  be  used  to  denote  the  normalized 
defects  per  unit  of  any  item,  whether  it  is  a  part,  assembly,  or  equipment.  The 
units  of  defect  density  are  defects  per  million  (DPM)  or  defects  per  unit  (DPU) 
as  appropriate.  The  term  defect  density  shall  be  used  throughout  this  report 
(and  the  recommended  revised  version  of  the  handbook)  in  favour  of  the  HDBK 
term  fraction  defective.  The  procedure  for  estimating  the  incoming  defects  is 
Procedure  A. 

Because  HDBK  defect  density  data  is  generic  and  may  not  be  sufficiently 
current  for  certain  programs,  the  recommended  HDBK  changes  contains  a 
methodology  for  determining  defect  densities  from  observed  factory  and  field 
data.  These  data  would  be  used  in  preference  to  the  HDBK  tables  provided 
sufficient  data  has  been  collected  and  validated. 

2.2.3  Estimating  Screening  Strength.  The  next  step  is  to  estimate  what 
fraction  of  the  latent  defects  is  removed  by  ESS  and  to  optimize  the  screen 
type,selection,  and  placement.  The  method  for  doing  this  is  described  in 
procedure  B  of  the  HDBK.The  terminology  used  in  the  current  HDBK  denotes 
the  strength  of  a  screen  as  Test  Strength,and  the  effectiveness  in  precipitating 
a  defect  as  screening  strength.  These  terms  can  possibly  lead  to  confusion  and 
inadvertent  misapplication;  thus,  this  report  and  the  recommended  changes  to 
the  handbook  redefine  these  terms.  With  the  revised  terminology,  a  screen  is 
used  to  remove  defects  by  precipitation  and  detection.  The  effectiveness  of  a 
screen  is  thus  measured  by  its  screening  strength  and  is  determined  from  the 
product  of  a  precipitation  and  a  detection  efficiency  term.To  apply  the 
procedure,  the  HDBK  user  determines  the  precipitation  efficiency  for  a  given 
stress  type,level,  and  duration  using  lopkup  tables  or  solving  equations  that 
are  provided  in  the  handbook.  The  test  detection  efficiency  is  determined  based 
on  the  type  of  testing  performed  and  whether  testing  is  performed  during 


exposure  to  stress.  The  screening  strength  is  then  determined  from  the  product 
of  these  factors.  To  improve  the  , accuracy  of  this  estimate  of  screening  strength, 
the  revised  HDBK  contains  a  methodology  for  determining  screening  strength 
from  actual  factory  and  field  data.  The  value  obtained  from  the  observed  data 
would  be  used  in  preference  to  the  HDBK  tables  provided  sufficient  data  has 
been  collected  and  validated. 

2.2.4  Failure  Free  Acceptance  Test.  A  failure  free  acceptance  test  (FFAT)  is 
designed  using  Procedure  C  and  has  the  purpose  of  demonstrating  that  the 
remaining  defect  density  is  adequate  for  the  desired  reliability.  The  FFAT  in 
itself  uses  ESS  to  precipitate  a  fraction  of  the  remaining  defects.  The  number 
of  defects  precipitated  and  removed  is  related  to  the  effects  re*raining  and 
thus  is  an  indicator  of  residual  defects  and  thus  reliability.  The  HDBK 
procedure  was  found  to  be  deficient  and  had  to  be  modified  to  be  implemented 
for  the  study.  The  reasons  for  the  changes  and  other  concerns  with  the 
procedure  are  discussed  in  2.9. 

2.2.5  Cost  Effectiveness  Analysis.  The  cost  effectiveness  analysis  used  as 
part  of  procedure  B  to  optimize  the  ESS  program  is  described  in  Procedure  D. 
In  principle  the  user  determines  the  total  cost  of  the  factory  ESS  program  and 
compares  the  cost  per  defect  removed  to  a  representative  cost  of  a  field  defect  ie 
$1000  that  is  called  the  threshold  cost.  The  intent  is  to  optimize  the  ESS 
program  based  on  the  combined  cost  to  the  producer  and  customer.  The 
implication  is  that  if  ESS  costs  the  producer  more  than  $1000  per  defect  then 
perhaps  a  portion  of  the  ESS  (hence  cost)  is  unwarranted  and  its  necessity 
should  be  determined  by  the  procuring  activity. 

2.2.6  Monitorinpr.  Evaluation  and  Control.  Monitoring,  evaluation,  and 
control  of  the  ESS  program  are  accomplished  using  quality  control  charts  and 
are  described  in  Procedure  E.  The  ESS  parameters  of  interest  are  initial  defect 
density  (Din)  and  TS  (now  called  SS)  with  potential  problems  being  identified 
by  these  parameters  being  either  higher  or  lower  than  expected.  An  inherent 
difficulty  however,  is  that  neither  of  these  parameters  are  directly  obseryahle 
by  the  user;hence,  methods  of  estimating  them  from  factory  fallout  are  also 
provided. 


2.3  Application  of  DOD-HDBK-344  procedures.  This  section  describes  the 
application  for  the  handbook  procedures  to  the  three  equipments  selected  for 
the  study  contract. 

2.3.1  Designing  ESS  Program  in  Retrospect.  In  order  to  apply  the  HDBK 
methodology,  it  was  necessary  to  design  the  ESS  program  in  retrospect.  This 
was  accomplished  by  applying  the  HDBK  methodology  to 

i)  estimate  screening  strength 

ii)  determine  initial  defects  per  system  prior  to  ESS 

iii)  determine  defects  removed  and  remaining  throughout  ESS 

2.3.1. 1  Step  1.  Determine  Screening  Strength..  The  screening  strengths  were 
computed  using  the  equations  that  were  used  to  create  DOD-HDBK-344  tables 
5.14  -  5.18  (courtesy  of  A1  Saari,  Hughes  Aircraft  Company  of  Fullerton, 
California).  The  use  of  equations  rather  than  look-up  tables  provides  greater 
flexibility  and  accuracy  since  the  tables  do  not  include  all  of  the  appropriate 
condUions  or  parameters.  For  example.  Equipment  A  assembly  level  ESS  uses 
50  temperature  cycles  whereas  Table  5.15  only  goes  as  high  as  12  cycles. 

The  screening  programs  for  the  equipment  (Part  A  Figure  4.1,  4.2,  and  4.3) 
were  thus  modeled  and  the  results  summarized  in  Figures  2.1,  2.2,  arid  2.3  for 
Equipment  A,  B,  and  C  respectively. 

2. 3. 1.2  Step  2,  Define  the  Complexity.  The  initial  defects  per  system  were 
calculated  by  multiplying  the  humber  of  parts  in  various  part  types  and 
reliability  grades  by  the  appropriate  defect  densities  found  in  the  HDBK 
Tables  5.2  to  5.13. . 

The  complexity  of  equipment  A,  B,  and  C  are  provided  in  Figures  2.4  through 
2.8.  The  complexity  of  equipment  B  has  been  d'iscribed  for  each  of  the  major 
systems,  i.e.,  MDI,  MDRI,  and  HUD. 
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Test  Level 

Environment 

Precipitation  Stress 
Efficiency  Parameters 

vaassss 

aaaaaaaaa 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

II 

-  II 
II 

Card-Module 

Temp  Cycle 

.99 

SOcycles , 1 5C/min , 
130  C  range 

Module 

System 

Rand  Vib 

.96 

12  gRMS,  10  min 

Pre  integration 

Temp  Cycle 

.65 

4  cycles,  5  C/min, 
125  C  range 

Integration 

Temp  Cycle 

.34 

2  Cycles, 4  C/min, 
122  C  range 

Rand  Vib 

.63 

6  gRMS,10  min  in  1 
axis 

Pre  ATP 

Temp  Cycle 

.34 

2  cycles,  4  C/min, 
122  C  range 

ATP 

Temp  Cycle 

.57 

4  cycles,  4  C/min, 
122  C  range 

Rand  Vib 

Notes: 

.35 

5.5  gRMS,5  min 
each  of  3  axis 

Temp  Cycle  =  Temperature  Cycling 

Rand  Vib  ®  Random  Vil?ration 

ATP  «  Acceptance  Test  Procedure 


Figure  2.1  EQUIPMENT  A  SCREENING  PROGRAM 
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Test  Level  En-ironment  Precipitation 

Efficiency  Stress  Parameters 


Card-Module  none 
System 

Temp  Cycle  .82 

Rand.Vib  .64 

Temp  Cycle  .97 

Sine  Vib  .01 

Temp  Cycle  .96 


Notes: 


Sine  Vib  .Oi 


8  cycles r  4  C/min, 
125  C  range 

6.1  gRMS,10  min 

17  Cycles, 4  C/min, 
125C  range 

2.2  g,  20  min 
IScycles,  4  C/min, 
125  C  range 
2.2  g,  20  min 


Temp  Cycle  =  Temperature  Cycling 

Rand  Vib  =  Random  Vibration 


Sine  Vib  =  Sine  Vibration 


Figure  2.2  EQUIPMENT  B  SCREENING  PROGRAM 


Test  Level  Environment  Precipitation 

Efficiency  Stress  Parameters 


Card-Module 

Temp  Cycle 

.99 

30  cycles, 15C/min, 
140C  range 

System 

Temp . Cycle 

.86 

10  cycles,  4 

C/min,  110  C  range 

Rand  Vib 

.63 

6  gRMS,  10  min 

Notes;  ' 

Temp  Cycle  =  Temperature  Cycling 
Rand  Vib  =  Random  Vibration 

FIGURE  2.3  EQUIPMENT  C  SCREENING  PROGRAM 


PART  TYPE 

RELIABILITY  GRADE 

MICROCIRCUITS 

B 

B1 

2&6 

256 

SEMICONDUCTORS 

TXV 

JAN 

Transistors 

147 

97 

Diodes 

302 

34 

PASSIVES 

P 

MIL 

Resistors 

1191 

25 

Capacitors 

663 

17 

OTHERS 

MIL 

MIL  EQUIV 

Magnetics 

5 

12 

Relays 

42 

0 

Connectors 

42 

'  0 

PHB 

28 

0 

CONNECTIONS 

Hand 

1340 

Wave  solder 

21900 

Wire  Wrap 

1250 

PIGORE  2.4  PARTS  and  WORKMANSHIP  COMPLEXITY  of  EQUIPMENT 
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PART  TYPE  RELIABILITY  GRADE 


MICROCIRCUITS 

BO 

B1 

438 

480 

SEMICONDUCTORS 

TXV 

JAN 

Transistors 

105 

18 

Diodes 

220 

4 

PASSIVES 

P 

MIL 
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1350 

16 
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935 

11 

OTHERS 
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,  34 

28 

Relays 

1 

0 

Connectors 

36 

4 
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23 

0 

CONNECTIONS 

Wave  solder 

20555 

FIGURE  2.5  PARTS  and  WORKMANSHIP  COMPLEXITY  of  EQUIPMENT 
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FIGURE  2 . 6  PARTS 
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B1 

57 
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14 
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PART  TYPE 


RELIABILITY  GRADE 


MICROCIRCUITS 

BO 

B1 

170 

174 

SEMICONDUCTORS 

TXV 

JAN 

Transistors 

105 

19 

Diodes 

312 

10 

PASSIVES 

P 

MIL 

Resistors 
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3 

Capacitors  ■ 

786 

14 

OTHERS 

MIL  MIL 
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Magnetics 

40 

55 

Relays 

4 

0 
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39 

4 
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20 

0 
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Hand 

35 

Wave  solder 

13106 

FIGURE  2.7  PARTS 

and  WORKMANSHIP  COMPLEXITY 

Of  EQUIPMENT 
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PART  TYPE  RELIABILITY  GRADE 


MICROCIRCUITS 

BO 

B1 

2 

425 

SEMICONDUCTORS 

TXV 

JAN 

Transistors 

192 

57 

Diodes 

.205 

34 

PASSIVES 

P 

M 

MIL 

Resistors 
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888 

79 

Capacitors 

840 

12 

24 

OTHERS 

MIL 

MIL  EQUIV 

Magnetics 

2 

30 

Relays 

3 

0 

Connectors 

2 

53 

PWB 

15 
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Hand 

132 

Wave  solder 

12729 

FIGURE  2.8  PARTS  and  WORKMANSHIP  COMPLEXITY  of  EQUIPMENT  C 
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Although  the  predicuon  of  factory  defect  densities  and  fallout  are  of  major 
interest  to  the  HDBK  user,  the  HDBK  does  not  currently  have  the 
methodology  or  data  base  necessary  to  perform  the  required  calculatioAs.  The 
handbook  recognizes  that  the  defect  density  is  related  to  the  stress  level  and 
therefore  has  defect  densities  for  the  various  application  environments  eg., 
AIF,  ML  etc.  The  problem  is  that  factory  ESS  has  stress  levels  difTerent  than 
the  environments  covered  ip  the  tables  and  further,  the  factory  ESS  is  not 
necessarily  directly  related  to  the  field  stress  levels.  The  objective  is  to 
stimulate  not  simulate  two  different  field  environments  could  conceivably 
have  the  same  factory  ESS  program.  This  deficiency  and  proposed  solutions 
are  discussed  in  2.5. 

The  results  of  these  calculations  are  provided  in  Table  2.1  for  a  selection  of 
different  application  environments. 

2.3.1.3  Step  3.  Determine  Defects  Removed  and  Remaining.  The  defects 
removed  by  ESS,  i.e.,  factory  fallout  were  estimated  by  multipl3dng  the  defects 
per  system  determined  in  2.3. 1.2  by  the  screening  strength  of  each  ESS  as 
determined  from  2.3. 1.1.  Defects  escaping  one  screen  carried  over  for 
subsequent  screens. 

As  a  result  of  the  calculations,  the  total  latent  defects,  and  the  detected  and 
undetected  patent  defects,  were  calculated  for  each  assembly  test  level.  Yields 
were  then  calculated  from  the  defects  removed  based  on  a  Poisson  distribution 
i.e.  Yield  =  EXP  (-  Detected  Patent  Defects). 


37 


TABLE  2.1  Predicted  Defects  PER  SYSTEM. 


SYSTEM 

PREDICTED  DEFECTS  PER  SYSTEM 

BASED  ON  CURRENT  DOD-HDBK-344 

AIF 

ML 

CL 

TOTAL 

% 

PARTS 

%  WRK 

TOTAL 

% 

PARTS 

WRK 

TOTAL 

■*> 

PARTS 

% 

WRK 

Equip¬ 
ment  A 

.95 

81 

20 

H 

77 

22 

31.9 

68 

32 

MDI 

.70 

93 

mm 

2.04 

92 

8 

32.9 

92 

8 

MDRI 

1.59 

97 

3 

1.9 

97 

3 

21.2 

95 

HUD 

.91 

96 

4 

2.86 

94 

4 

36:6 

95 

Equip¬ 
ment  C 

1.09 

96 

H 

2.9b 

95 

5 

32.7 

93 

8 

2.4  Collection  of  Observed  Factory  Defect  Data.  Prior  to  this  study  contract, 
LSL  had  developed  an  extensive  and  computerized  Failure  Reporting, 
Analysis,  and  Corrective  Action  system  (FRACAS)  and  data  base  that 
contained  test  and  failure  data  on  the  three  equipment  selections.  An 
v^stablished  data  base  of  actual  production  failures  was  thus  available  for  this 
project  and  was  used  to  compare  the  actual  factory  results  with  the  HDBK 
predictions  for  both  defect  density  and  screening  strength.  This  section 
describes  the  collection  and  preliminary  analysis  of  this  data. 

2.4.1  System  Level  Data.  The  system  level  removal  data  were  analyzed  to 
determine  the  nature  and  cause  of  the  test  discrepancy;  thus,  for  system  test 
level,  it  was  possible  to  distinguish  between  "removals”  and  actual  defects  and 
to  also  identify  the  probable  cause  eg  part, workmanship,  design  etc.  This 
review  constituted  a  "preliminary"  review  as  described  in  4.5.3  of  the  proposed 
HDBK  (see  Appendix  B)  and  was  performed  in  advance  of  detailed  root  cause 
physics  of  failure  analyses.  Detailed  analyses  were  performed  when  problems 
and  out  of  control  conditions  were  identified. 
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The  need  for  detailed  analysis  was  determined  by  comparing  actual  results 
with  established  requirements  and  expectations  that  were  directly  relatable  to 
reliability  and  cost  requirements.  The  method  logy  used  by  LSL  for 
determining  the  need  for  detail<‘i  analyses  has  been  included  in  the 
recommended  revisions  to  the  HDBK.  Because  of  the  available  data,  it  was 
possible  to  conduct  part  of  the  study  contract  analyses  by  writing  special 
software  to  extract  the  data  from  FRACAS  that  was  pertinent  to  the  ESS 
analysis.  The  scope  of  this  study  contract  did  not  allow  for  additional  root 
cause  failure  analysis. 

2.4.2  Presvstem  Level  Data.  Although  all  presystem  level  test  and  failure 
data  were  also  recorded  in  FRACAS,  the  preliminary  review  that  was 
performed  on  system  level  discrepancies  was  only  performed  at  assembly  and 
module  level  when  trends  and  potential  problems  were  indicated.  Thus,  the 
data  base  for  lower  leva!  crntained  a  mixture  of  defect,  as  well  as  repair  and 
removal  data.  Patent  and  latent  defects  were  thus  included  with  patent-error 
defects  as  well  as  indncad  and  diagnostic  removals  etc.  Since  data  in  this  form 
was  not  directly  saita^!  •*  ter  ESS  analysis  purposes,  some  method  of  extracting 
latent  defects  was  reqv  lred.  Because  of  the  high  reliability  emphasis  on  the 
Equipment  A  prog.'‘am,  ^^tensive  analysis  of  preassembly  removals  and 
defects  had  been  perfd-rned  over  several  years.  This  data  was  used  to  provide  a 
statistical  inference  ar  <>;  what  percentage  of  assembly  removals  were  likely  to 
be  latent  defects.  Thii  factor  was  then  applied  to  the  raw  assembly  level  data 
to  estimate  the  number  of  latent  defects  on  all  three  programs. 

The  primary  need  for  lower  level  data  was  to  assess  assembly  level  ESS 
effectiveness  and  also  to  estimate  and  validate  the  defect  density  estimates  for 
the  various  commodity-reliability  grades.  The  error  introduced  by  the  use  of  a 
factor  to  determine  assembly  level  latent  defects  from  removal  data  was 
considered  in  the  analysis  process  and  was  not  believed  to  be  consequential  to 
the  results  and  conclusions.  Where  the  results  could  have  affected  the  HDBK 
accuracy,  the  appropriate  parameter  was  indicated  as  being  "user-definable” 
and  provided  as  a  "suggested  value”  in  the  HDBK  revisions. 
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2.5  Analysis  of  observed  Predicted  Results.  This  section  compares  the  HDBK 
predictions  for  defect  density  and  screehing  strength  with  the  observed  results 
and  discusses  recommended  changes  to  the  ESS  methodology!  The  analysis 
addresses  defect  density  in  2.5.1  and  screening  strength  in  2.5.2. 

2.5.1  Comparison  of  observed  and  estimated  defects.  The  observed  factory 
defects  from  system  level  ESS  are  compared  with  the  HDBK  estimates  in 
Table  2.2  and  2.3.  Recall  that  since  the  HDBK  did  not  have  a  methodology  or 
data  base  for  predicting  factory  ESS  defect  densities,  the  predictions  were 
made  for  several  environments.  The  selection  of  the  particular  environments 
for  these  tables  was  not  based  on  the  eventual  application  of  the  equipment, 
since  this  has  no  meaning  for  factory  ESS,  but  because  they  provided  the 
closest  correlation  with  actual  results,  and  would  thus  be  useful  for  rescaling 
purposes. 

The  inability  to  predict  factory  defect  levels  represents  a  major  modeling 
problem  that  must  be  corrected  to  have  a  viable  ESS  methodology.  The' 
recommendation  for  correcting  this  was  to  rescale  the  HDBK  tables  to  create  a 
reference  for  factory  ESS.  Since  defects  are  a  function  of  stress  level,  as 
indicated  by  including  the  various  environmental  conditions  in  the  HDBK 
tables,  a  stress  adjustment  factor  is  also  required  to  modify  the  factory 
estimate.  However,  before  addressing  this  problem  an  additional  fundamental 
problem  had  to  be  addressed.  The  proportions  of  workmanship  and  parts 
defects  in  the  observed  data  did  not  correlate  with  the  HDBK  predicted, 
proportions.  The  problem  was  addressed  by  analysing  workmanship  and  part 
defect  density  estimates  separately. 
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TABLE  2.2  Observed  Defects  Per  System 


OBSERVED  DEFECTS  PER  SYSTEM  (SYSTEM  LEVEL 
TESTING) 

SYSTEM 

TOTAL 

PARTS 

WORKMANSHIP 

»  PARTS 

% 

WORKMA.VSHtP 

Equipment  A 

1.44 

1.09 

35 

76 

24 

Equipment  B 

MDI 

2.45 

1.82 

.63 

74 

26 

MDRI 

1.18 

.86 

.32 

73 

27 

HUD 

1.52 

1.02 

.5 

67 

33 

Equipment  C 

2.54 

2.11 

.43 

83  , 

17 

PREDICTED  DEFECTS 


SYSTEM 

PREDICTED  DEFECTS  PER  SYSTEM 

- 

AIF 

ML 

CL 

TOTAL 

« 

PARTS 

% 

WRK 

TOTAL 

* 

PARTS 

« 

WRK 

TOTAL 

«' 

PARTS 

% 

WRK 

Equipment  A 

.95 

81 

20 

77 

22 

31.9 

68 

32 

Equipment  B 
MDI 

.70 

,93 

■ 

2.04 

92 

8 

32.9 

92 

8 

.MDRI 

1.59 

97 

H 

1.9 

97 

B 

21:2 

95 

5  , 

HUD 

.91 

96 

a 

2.86 

94 

B 

36.6 

95 

5 

Equipment  C 

1.09 

96 

4 

2.95 

95 

5 

32.7 

93 

8 
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OBSERVED 
DEFECTS  PER 
SYSTEM 

PREDICTED  DEFECTS  PER 
SYSTEM  AIF  ENVIRONMENT 

PREDICTED  DEFECTS  PER 
SYSTEM  ML  ENVIRONMENT 

PREDICTED 
DEFECTS  PER 
SYSTEM 

PREDIC¬ 

TION 

ERROR 

PREDICTED 
DEFECTS  PER 
SYSTEM 

PREDIC¬ 

TION 

ERROR 

1.44 

0.95 

-52% 

2.7 

474b 

2.45 

0.7 

-250% 

2.04 

-204b 

1.18 

1.59 

264b, 

1.9 

38% 

1.52 

0.91 

-674b 

2.86 

47% 

2.54 

1.09 

-1334b 

2.95 

14% 

i)  Analysis  and  recommendations  for  predicting  workmanship  defects 

The  error  between  the  predicted  and  observed  defects  per  system  is  shown  in 
Table  2.3.  The  prediction  error  was  not  consistent  across  all  equipment  types 
for  either  the  AIF  or  ML  environments,  thus  indicating  there  was  a 
fundamental  prediction  problem  other  than  rescaling  the  handbook  tables. 
Examination  of  the  data  in  Table  2.2  showed  an  inconsistency  in  the 
proportion  of  workmanship  and  parts  related  defects.  Subsequent  analysis 
revealed  that  the  discrepancy  was  due  primarily  to  the  estimate  of 
workmanship  defects. 

The  observed  fractions  of  parts  and  workmanship  defects  for  Equipment  A 
were  76%  parts  24%  workmanship  and  compared  favourably  with  the  77% 
parts  23%  workmanship  predicted  for  ML. 

The  same  proportions  for  Equipment  B  were  70%  parts,  30%  workmanship 
observed  vs.  95%  parts,  5%  workmanship  predicted.  For  Equipment  C,  the 
values  were  83%  parts  17%  workmanship  observed,  compared  with  95%  parts, 
5%  workmanship  predicted.  The  predictions  for  both  Equipment  B  and 
Equipment  C  were  thus  unsatisfactory. 

A  closer  examination  revealed  that  workmanship  defects,  and  hence  the  parts 
vs.  workmanship  proportions,  were  greatly  influenced  by  the  number  of  hand 
solder  connections.  This  was  due  to  the  HDBK  defect  density  factor  for  hand 
solder  connections  being  nearly  40  times  higher  than  that  for  reflow/wave 
soldering. 

Since  Equipment  A  contains  a  mixture  of  both  flat  pack  (hence  hand  solder) 
and  through  hole  (hence  wave  soldered)  parts,  conipared  with  essentially  all 
wave  soldered  connections  on  Equipment  B  and  Equipment  C,  the  proportions 
of  workmanship  defects  appear  quite  different. 

Further  error  was  introduced  because  the  handbook  only  considered  solder 
connections  in  the  estimate  of  workmanship  defects.  In  practice,  many 
workmanship  faults  were  related  to  other  factors,  for  example: 

i)  lead  forming  and/or  dressing. 
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ii)  insulation  stripping, 

iii)  shorts  due  to  foreign  materials, 

iv)  component  mounting  and  bonding, 

v)  mechanical  handling  damage, 

vi)  electrical  (eg.  ESD)  handling  damage, 

vii)  wire  routing, 

viii)  mechanical  assembly  eg.  torquing, 

ix)  bending  connector  pins. 

Since  there  was  insufficient  data  to  determine  appropriate  defect  densities  for 
the  all  of  the  various  workmanship  considerations,  the  recommendation  was  to 
use  a  different  methodology  for  determining  workmanship  complexity.  The 
recommended  approach  was  to  adopt  the  DOD-STD-2000,  methodology  for 
determining  assembly  and  soldering  complexity.  The  approach  is  described  in 
Appendix  C  of  DOD-STD-2000-1  and  in  principle  determines  an  assembly 
normalizing  number  based  on  the  niunber  of  components  +  leads  -4-  terminals 
+  wire  +  PWB  and  a  soldering  normalizing  number  based  on  the  number  of 
solder  connections.  Although  the  purpose  for  determining  these  values  is 
different  for  DOD-STD-2000  than  DOD-HBDK'344,  (also  that  DOD-STD-2000 
became  MIL-STD-2000  during  the  study  program)  the  means  of  describing 
workmanship  complexity  seemed  to  be  simple  and  valid.  If  more  industry  data 
becomes  available,  it  would  be  possible  to  define  workmanship  complexity 
more  precisely  in  terms  of  the  number  of  individual  processes  and  types; 
however,  in  the  interim  ^e  DOD-STD-2000  approach  should  be  sufficiently 
accurate  for  ESS  planning  purposes. 

To  establish  defect  densities  for  the  assembly  and  soldering  normalizing 
numbers,  the  observed  workmanship  defects  per  system  were  compared  with 
recalculated  normalizing  numbers  for  each  of  equipments.  The  defect 
densities  were  thus  determined  to  be  28  PPM  for  tUb  assembly  normalizing 
number  and  6  PPM  for  the  soldering  normalizing  number.  Before  assessing 
how  appropriate  this  modeling  was,  the  problem  of  estimating  part  defect 
density  still  had  to  be  addressed. 
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b)  Analysis  and  recommendations  for  part  defects.  The  Equipment  A 
FRACAS  data  extraction  program  was  modified  to  provide  the  capability  of 
determining  the  defect  density  of  the  system  level  fallout  by  part  type  and 
reliability  grade.  Using  this  facility,  it  was  thus  possible  to  determine  the 
defect  densities  of  various  commodities  and  to  compare  them  to  the  HDBK 
defect  density  tables  ie  HDBK  Tables  5.2  -  5.10. 

Before  making  the  comparison.it  was  first  necessary  to  address  the  problem  of 
not  being  able  the  estimate  defects  and  fallout  under  different  and  varying 
stress  levels  such  as  occur  in  factory  ESS.  The  solution  to  this  problem 
essentially  modified  the  modelling  ahd  mathematics  to  includes  a  stress 
adjustment  factor  (SAF)  and  defined  defects  relative  to  a  'baseline'  stress.  The 
SAF  factor  rescales  defects  densities  defined  at  this  baseline  stress  to  any 
desired  factory  ESS  or  field  application.  Since  the  appropriate  SAF  was  not 
known  at  this  stage  in  the  analysis,  the  defect  densities  were  compared 
ratiometrically  as  a  group  within  one  stress  level' and  collectively  between  two 
stress  levels  ie  ML  application  and  factory  ESS  stress  levels  for  Equipment  A. 

The  defect  densities  from  the  HDBK  tables  for  an  ML  application  and  the 
observed  defect  densities  were  normalized  with  respect  to  MIL  M  38510 
integrated  circuits  and  then  compared  on  a  ratiometric  basis.  The  ML 
environment  was  selected  since  it  provided  a  reasonably  close  correlation  with 
the  observed  results.  The  defect  density  for  semiconductors  was  calculated  as  a 
weighted  average  of  the  transistor  arid  diode  tables.  This  analysis  was 
designed  to  disclose  any  anomalies  in  the  defect  density  of  one  part  type  with 
respect  tc  the  other  part  types.  As  can  be  seen  from  Table  2.4,  the  greatest 
errors  occurred  on  capacitors  and  magnetics  where  the  error  was  approaching 
an  order  of  magnitude.  The  other  part  types  correlated  within  a  factor  of  2. 
This  later  degree  of  correlation  was  considered  satisfactory  knowing  that  the 
Tables  represent  a  general  average  and  that  some  spread  should  be  anticipated 
considering  the  wide  diversity  in  technologies  and  vendors  within  any  group. 
Although  based  on  these  findings  the  HDBK  tables  for  capacitors  and 
magnetics  should  probably  be  rescaled,  the  HDBK  revisions  recommended  in 
Appendix  B,  retain  the  original  values  (ratiometrically)  since  the  Equipment 
A  data  base  alone  was  not  considered  to  be  sufficient  to  make  a  generalized 
change. 
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However,  the  methodology  of  the  proposed  HDBK  was  changed  to  allow  the 
user  to  analyze  and  use  his  own  data,  thereby  reducing  the  need  for  high 
accuracy  in  the  HDBK  tables  and  providing  the  methodology  for  the  user’s 
data  to  override  any  anomalies  in  the  HDBK  data. 

To  compare  the  magnitude  of  the  defect  densities,  the  HDBK  defect  densities 
for  the  ML  environment  were  compared  with  the  observed  values.  The  purpose 
of  this  comparison  was  to  determine  an  appropriate  scaling  factor  for  creating 
a  factory  ESS  defect  density  table  from  the  HDBK  data.  As  shown  by  the  last 
column  of  Table  2.3,  the  ML  environment  can  be  rescaled  by  1.15  to  create  a 
defect  density  data  base  for  system  level  fallout.  The  factor  of  1.15  however  is 
incomplete  since  it  is  necessary  to  model  total  defects  not  just  system  level 
removals.  It  is  thus  necessary  to  convert  system  level  fallout  into  total  defects 
by  dividing  the  system  level  fullout  by  the  screening  strength  (i.e.  to  compute 
total  latent  defects  preseat  at  system  level)  and  adding  the  defects  removed  at 
assembly  level. 
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TABLE  2.4.  COMPARISON  OF  OBSERVED  AND  DOD  HDBK-344 

DEFECT  DENSITIES  FOR  PARTS 


PART 

OOD-HDBK.344 

EQUIPA  FACTORY  DATA  1987  4  1988 

HDBK-344 

COMMODITY 

ML  PPM 

RATIO  TO  STOIC 

PPM 

RATIO  TO  STOIC 

ADJUSTMENT 

FACTOR  .  . 

STD  IC 

139 

1.0 

194 

1.0 

1.40 

.\0.\  STD  IC 

279 

2.0 

587 

3.0 

2.10 

STD  SEMI 

NON  STD 

220 

1.6 

164 

0.8 

0.74 

SEMI 

881 

6.3 

1178 

6.1 

1.34 

RESISTORS 

54 

0.4 

11'6 

0.6 

2.16 

CAPACITORS 

415 

3.0 

70 

0.4 

0.17 

MAGNETICS 

5643 

40.6 

767 

4.0 

0.14 

AVG.  1.15 

The  fallout  from  assembly  level  ESS  on  Equipment  A  was  reviewed  to 
determine  the  removal  rates  for  parts,  and  workmanship  defects  and  errors. 
(To  understand  and  appreciate  these  defect  definitions  and  the  importance  in 
separating  latent  and  patent-error  defects,  the  reader  is  referred  to  Appendix 
A).  The  root  cause  failure  analysis  data  in  FRACAS  was  then  used  to 
determine  the  ratio  of  latent  defects  to  removals  and  errors.  The  results 
indicated  that  approximately  25%  of  the  reported  removals  at  system  level 
where  actually  caused  by  latent  defects.  Considering  these  factors  and  the 
estimated  system  level  screening  strength,  the  rescaling  factor  was  estimated 
as  being  1.5.  Since  the  Equipment  A  ESS  Stress  levels  (not  necessarily 
duration)  were  sufficiently  close  to  the  R&M  2000  stress  level  guidelines,  the 
defect  densities  for  factory  ESS  were  consistent  with  R&M  2000  guidelines 
were  determined  by  rescaling  the  ML  defect  densities  found  in  the  HDBK 
Tables  5.2  through  5.10  by  the  1.5x  factor. 
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c)  Conclusions  concerning  the  changes  to  defect  density  modelling  and  tables. 
The  final  task  in  the  analysis  and  recommendations  concerning  defect 
densities  was  to  recompute  the  predicted  defect  fallout  based  on  the  modelling 
and  defect  density  table  changes  described  above  and  to  compare  these  results 
to  the  actuals  for  the  three  equipment  types.  The  comparison  was  made  for 
both  the  estimate  of  total,  part ,  and  workmanship  related  defects.  The  effect 
that  the  limited  sample  size,  ie  number  of  systems  produced  had  on  the 
observed  results  was  also  considered .  The  observed  mean  can  be  expected  to  be 
different  than  the  actual  mean  defect  density  because  of  statistical  variation. 
This  variaticq  was  calculated  from  the  standard  error  of  the  mean,  assuming 
the  defects  were  Poisson  distributed.  The  results  of  the  comparison  are  given  in 
Table  2.5.  The  upper  part  of  the  table  compares  the  magnitude  of  the  predicted 
and  observed  total  defects.  This  suggests  the  predicted  results  were  within 
20%  of  the  observed  values  and  within  1  to  2  standard  deviations  of  the 
statistically  expected  error  .  The  average  error  across  5  equipments  was  1.2%. 
The  middle  part  of  the  table  compares  the  parts  defects  per  system.  This 
indicates  that  the  estimate  for  part  defects  was  within  30%  with  an  average 
error  of  0.0%.  The  lower  part  of  the  table  compares  the  estimated  and 
observed  workmanship  defects.  The  error  is  effectively  within  31%  with  an 
average  of  0.0%  across  the  5  equipments.  The  recommended  changes  to  the 
HDBK  thus  appear  to  have  improved  the  accuracy  problems  encountered  with 
the  current  handbook.  The  simplified  workmanship  complexity  definition 
based  upon  DOD  STD  2000  also  seems  to  be  sufficiently  accurate  for  planning 
purposes. 
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Table  2.5  Comparison  of  Observed  and  Predicted  Defects  per  System. 


■ 

TOTAL  DEFECTS  PER  SYSTE M 

' 

PREDICTED 

DEF/SYS 
(REVISED  HDBK) 

OBSERVED  DEFECTS 

PREDICTION 

STANDARD 

EQUIPMENT 

DEF/SYS 

ERROR 

ERROR 

TYPE 

SYS  ALL  LEVELS 

OF 

MEAN 

Equipment  A 

a.4 

1.44 

2.10 

10% 

4% 

Equipment  B 

MDI 

2J1 

2.45 

2.45 

.6% 

14% 

MDRI 

1.03 

1.18 

1.18 

-14% 

11% 

HUD 

IJl 

1J2 

1.52 

20% 

12% 

'  Equipment  C 

3SI 

2.54 

3E1 

AVG. 

-4% 

IJK 

4% 

WORKMANSHIP  DEFECTS  PER  SYSTEM 


EQUIPMENT 
'  TYPE 

PREDICTED 

DEF/SYS 
(REVISED  HDBK) 

OBSERVED  DEFECTS 
DEF/SYS 

ALL LEVELS 

PREDICTION 

ERROR 

STANDARD 

ERROR 

OF 

MEAN 

Equipment  A 

0.714 

0.518 

27% 

8.2% 

Equipment  B 

. 

MDI 

0.783 

J37 

10% 

27  J% 

MDRI 

0J43 

JIO 

•31% 

21.2% 

HUD 

0.510 

0JO2 

2% 

2ej% 

Equipment  C 

0.535 

0.648 

AVG. 

-17% 

0.0% 

9.7% 

PARTS  DEFECTS  PER  SYSTEM 


EQUIPMENT 

TYPE 

PREDICTED 

DEF/SYS 

(REVISED  HDBK) 

OBSERVED  DEFECTS 

DEF/SYS 

ALL LEVELS 

PREDICTION 

ERROR 

STANDARD 

ERROR 

OF 

MEAN 

Equipment  A 

1.68 

1.642 

5% 

Equipment  B 

BIDI 

US 

U13 

•10% 

18% 

0.79 

0R81 

•9% 

13% 

HUD 

1.4 

.018 

27% 

15% 

Equipment  C 

3.12 

.162 

•1% 

4% 

AVG.  . 

0.0% 

®  ,  LATENT  DEF  ESTIMATED  AS  SYSTEM  DEF  X  1.5  FOR  EQUIP  A  AND  C  BECAUSE  OF  ASSY  T/C  SCREENING. 

LATENT  DEFECTS  =  SYSTEM  DEFECTS  FOR  (MDI.  MDRI.  HUD.  i.e.,  NO  ASSY  SCREENING! 

®  PREDICTION  ERROR  =  (Predicte<i/Actual-l)tOO% 

®  STANDARD  ERROR  OF  MEAN  DUE  TO  UMITED  SAMPLE  SIZE.  CALCULATED  ASSUMING  A  POISSON  DIST„ 
i.e,  5  =  SQUARE  ROOT  OF  (DEFECTS  PER  SYSTEM«YSTEMS  PRODUCED)  EXPRESSED  AS  A  PERCENTAGE 
OFTHEMEAN. 
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2.5.2  Analysis  of  Observed  and  Predicted  Screening  Strength.  The  basic 
definition  of  screening  strength  SS  (as  suggested  in  2.2.2  aboye)  is  the  fraction 
of  latent  defects  precipitated  and  removed  by  a  screen.  Screening  strength  is 
^us  the  product  of  a  precipitation  efficiency  term  and  a  detection  e^ciency 
term.  In  order  to  measure  the  screening  strength  of  the  actual  factory  ESS 
programs  used  on  the  three  equipment  selections,  the  effects  of  precipitation 
and  detection  need  to  be  individually  measured  and  assessed.  Further,  to  be 
able  to  compute  SS  both  the  latent  defects  removed  and  the  total  latent  defects 
present  need  to  be  determined.  Although  it  is  possible  to  separate  the  fallout 
into  errors  and  defects  by  performing  a  preliminary  analysis  (as  is  done  in  the 
LSL  FRACAS  system),  it  would  still  be  necessary  to  distinguish  between 
patent  defects  escaping  from  a  previous  screen  and  those  precipitated  by  the 
screen  being  analyzed.  Also,  the  total  latent  population  can  not  be  determined 
until  all  latent  defects  have  been  removed.  This  would  require  extensive 
factory  and  field  data  to  be  collected  and  analyzed  to  separate  latent  defects 
from  induced  defects  and  from  the  defects  expected  when  the  equipment  has 
reached  the  screening  limit  (ie  constant  failure  rate).  Even  then  the 
appropriate  value  for  total  latent  defects  would  still  not  be  known  because  of 
the  (probable)  effect  that  latent  defects  are  a  function  of  the  stress  level  and 
thus  the  change  in  stress  between  factory  and  field  could  significantly  affect 
the  perceived  total.  Performing  tests  before  and  after  a  screen  as  well  as 
performing  100%  analysis  to  root  cause  would  be  useful  in  an  overall  study 
contract  and  could  help  the  problem  of  determining  escaping  patent  defects, 
but  both  represent  an  undesirable  and  unwarranted  production  expense  and 
were  beyond  the  scope  of  this  contract.  Further,  performing  these  tests  and 
analyses  would  not  have  addressed  the  problem  of  being  able  to  determine  the 
total  latent  defects.  Some  method  of  being  able  to  measure  the  screening 
strength  needed  to  be  devised. 
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The  approach  adopted  for  the  study  was  to  curve  fit  the  production  data  to  a 
theoretically  expected  expression  and,  provided  the  data  were  reasonably  well 
represented  by  the  solution, extract  the  required  information  on  screening 
strength  from  the  curvefitting  parameters.  The  underlying  theory  and 
approach  taken  are  discussed  in  Appendix  A.  To  perform  the  analysis  care 
must  taken  in  the!  treatment  of  the  different  types  of  defects  ie  it  is  vital  that 
errors  be  distinguished  from  defects  and  that  defects  are  considered  as  latent, 
patent  and  a  function  of  applied  stress.  The  analysis  must  also  be  aware  of  the 
convoluted  effects  of  defect  precipitation  and  detection.  To  analyze  the  data,  it 
was  necessary  to  curvefit  various  combinations  of  data  to  determine  the 
distribution  in  normalized  defects  eg  0PM  or  DPU  with  respect  to  stress 
duration  eg  time  or  cycles.  These  analyses  were  performed  using  specially 
developed  software  and  curve  fitting  the  observed  results  to  the  following 
expression. 
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Dp=  EXISTING  PATENT  DEFECTS  AT  START  OF  ESS 
Di  =  LATENT  DEFECTS  AT  START  OF  TEST 
Dc  =  UMITING  LE.  CONSTANT  FAILURE  RATE’ 

De=  test  DETECTION  EFFICIENCIES 
B  =  TEST  DETECTION  EFFICIENCY 'K’FACTORCS) 
A  =  ESS  DEFECT  PRECIPITATION  TC  FACTOR 


This  equation  has  3  distinct  tenns  representing  l^e: 

a)  detection  of  previously  precipitated  latent  defects 

b)  detection  of  latent  defects  precipitating  during  ESS. 

c)  detection  of  defects  precipitating  at  a  constant  rate  i.e.,  determined  by 
the  limiting  MTBF. 

These  three  terms  and  the  combined  defect  removal  rate  are  illustrated  in 
Figure  2.9. 


The  detection  of  precipitating  defects  has  within  itself  two  exponential  terms, 
one  due  to  precipitation  efficiency  and  the  other  due  to  detection  efRciency. 

The  perceived  effect  of  precipitation  efficiency  and  detection  efficiency  depends 
upon  the  relative  'K’  factors  of  these  terms  and  is  illustrated  in  Figure  2.10. 
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FIGURE  2.9  ESS  Model. 
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EFFECT  OF  DIFFERENCES  IN  K  FACTORS 


If  the  K  factors  are  different,  the  result  is  a  nearly  exponential  distribution 
that  is  dominated  by  the  smaller  ’K’.  If,  for  example,  test  detection  is  ’weaker’ 
than  defect  precipitation,  the  measured  screening  strength  will  actually  be  the 
test  detection  efficiency.  In  many  instances,  i.e.,  when  there  is  a  significant 
difference  in  K  factors  the  general  expression  can  be  simplified  to 


'removed 


Dp  +  D^a  -«"*')  + 


(2-2) 


where  K  is  the  lesser  of  Ka  and  Kb  and  De  is  the  detection  efficiency  (assumed 
equal  for  all  3  terms). 


The  methodology  used  by  LSL,  and  discussed  herein,  was  to  curve  fit  actual 
test  data  to  the  theoretical  expression  given  above. 

With  this  method,  the  total  latent  defect  population,  escaping  patent  defects, 
defects  due  to  a  constant  failure  rate  (CFR),  and  screening  strength 
(precipitation  efficiency)  i.c.,  K  can  all  be  measured.  Note  that  if  the  K  is 
dominated  by  precipitation  effects,  the  detection  efficiency  (DE)  has  the  effect 
of  a  scaling  factor  and  does  affect  the  general  shape  of  the  curve.  Thus,  even 
though  DE  may  not  be  known,  the  curvefitting  methodology  ?s  still  valid  for 
determining  K  and  thus  the  precipitation  effectiveness. 

By  isolating  on  a  particular  environment,  eg.  T/C,  this  method  also  determines 
the  interaction  among  screening  environments  and  since  escaping  patent 
defects  are  measured,  it  is  also  possible  to  estimate  test  detection  efficiency 
(partial  factor).  Since  detection  efficiency  is  unique  to  a  specific  program 
which  precipitation  efficiency  is  generic,  only  the  precipitation  efficiency  was 
of  interest  for  the  study  and  is  addressed  in  this  report. 

For  the  analysis,  only  parts  and  workmanship  defects  were  used  with  defects 
caused  by  designs,  etc.,  excluded  to  avoid  possible  distortioti  of  the  results.  The 
results  of  the  analysis  are  discussed  in  a,b,c,  and  d  following. 

a)  Analysis  of  Equipment  A  Factory  Test  Data.  Equipment  A  ESS  test  flow  is 
described  in  Part  A  and  f  igure  2.1  and  consists  of  temperature  cycling 
interspersed  with  random  vibration.  Slight  changes  to  the  ESS  were  made 
over  the  course  of  the  study  and  are  reflected  in  the  analysis.  The  distribution 
ofEquipment  A  defects  as  a  function  of  elapsed  time  is  shown  in  Figure  2-11. 
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This  curve  is  of  initial  interest  in  that  it  curve  fits  to  the  exponential 
distribution  that  is  fundamental  for  ESS.  This  exponential  distribution  was 
also  confirmed  on  the  detailed  analysis  discussed  in  the  following. 

This  distribution  of  defects  is  broken  down  into  more  detail  in  Figure  2.12, 
wherein  the  defects  removed  at  each  test  are  indicated.  Temperature  cycling 
defects  are  annotated  by  T’  and  random  vibration  defects  are  annotated  by  'R’. 
The  elapsed  time  was  synthesized  from  the  test  duration. 

The  distribution  of  defects  detected  during  environmental  testing,  i.e., 
temperature  cycling  and  random  vibration  as  opposed  to  ambient  testing  is 
shown  in  Figure  2.13  against  temperature  transitions. 

If  the  defects  dua  to  random  vibration  are  excU'ded,  the  distribution  of  TC 
defects  is  as  shown  in  Figure  2.14.  The  analysis  results,  as  indicated  in  the 
figure  are: 

i)  total  population  of  temp  cycle  defects  =  .82  of  which  .80  are  latent  and 
.02  are  patent 

ii)  screening  strength  of  a  temperature  transition  (i.e.  3/4  of  temperature 
cycle)  =  .156 

i.e.  SS  =  1-Exp(-Kt )  where  K  =  .17,  t  =  1 

and  the  screening  strength  of  1  complete  cycle  (t  =  4/3)  is  thus  —  .288 

iii) the  limiting  MTBF  (during  T/C)  is  2500.  (Note  that  additional  data 
beyond  that  available  would  be  required  to  establish  this  value  with 
reasonable  certainty  and  that  the  measured  value  for  limiting  MTBF  is 
also  for  the  higher  stress  levels  of  factory  ESS  and  is  thus  a  lower  value 
that  would  be  experienced  in  the  field. 

A  significant  observation  is  that  R/V  defect  removal  has  not  perturbed  the 
curve;  thus,  indicating 

i)  RV  Defects  are  from  a  separate  defect  population  than  the  TC  defects 

ii)  TC  subsequent  to  RV  does  not  screen  or  detect  additional  defects. 


56 


T=r  TEMP  CYCLE 


R  =  RANDOM  VIBRATION 


FIGURE  2.12  Equipment  A  System  Level  Defects  vs.  Stress  Type 
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FIGURE  2.13  Equipment  A  Combined  Temp  Cycle  and  RV  Defect 

Distribution. 
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FIGURE  2.14  Equipment  A  TC  Defect  Distribution  1987 
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These  observations  are  made  on  the  basis  that  had  there  been  an  RV-TC 
interaction,  there  would  be  a  marked  point  of  inflexure  on  the  TC  defect 
distribution  curve. 

The  distribution  of  defects  detected  during  RV,  in  isolation,  is  shown  in 
Figure  2.15. 

Conclusions  from  this  data  are: 

i)  latent  (RV)  defect  population  =  .3 

ii)  screening  strength  of  RV  (5  min.,  5.5  grms)  =  .33  in  the  sensitive  axis 
(i.e.  l-Exp(-Kt),  where  K  =  .08,  t.  =  5) 

iii) RV  in  the  non  sensitive  axis  (i.e.,  perpendicular  to  the  plane  of  the 
boards)  is  not  effective  as  evidenced  by  the  lack  of  removals  for  Y  and  Z 
RV  axes. 

Combining  the  TC  and  RV  results  indicated  that  the  total  TC  and  RV  defect 
population  was  0.82  +  0.3  =  1.12  defects  and  that  RV  sensitive  defects 
accounted  for  .3/1.12  or  27%  of  the  total. 

Equipment  A  data  for  a  different  time  frame  was  also  analyzed  and  the  results 
are  provided  in  Figures  2.16  through  2.17.  The  comparison  of  the  results  for 
the  different  time  frames  is  discussed  in  d  hlong  with  the  results  for  equipment 
B  and  C. 

b)  Equipment  B  Screening  Strength  Analysis.  The  contractually  imposed 
Equipment  B  test  flow  is  described  in  Part  A  and  Figure  2.2,  and  consists  of 
random  vibration  followed  by  a  succession  of  temperature  cycling  plus  sine 
vibration  screens. 

Whereas  the  Equipment  A  test  flow  consisted  of  sequences  of  temperature 
cycling  (T/C)  interposed  with  random  vibration  (RV),  the  Equipment  B  flow, 
subsequent  to  RV,  is  primarily  continuous  temperature  cycling  (assuming  a 
negligible  screening  contribution  from  sine  vibration).  The  test  flow  is  thus 
useful  in  being  able  to  study  T/C  defect  precipitation  but  has  limited  use  for 
studying  RV  defect  precipitation. 
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Using  analysis  techniques  described  above,  the  cumulative  defect 
distributions  for  Equipment  B  are  provided  in  Figures  2.18  through  2.19. 

The  data  analysis  is  similar  to  Equipment  A  and  will  be  discussed  in  c 
following. 

c)  Equipment  C  Screening  Strength  Analysis.  The  test  flow  for  Equipment  d 
\yas  previously  described  in  Part  A  and  Figure  2.3,  and  consisted  of 
temperature  cycling  interposed  with  one  RV  sequence. 

The  test  flow  can  thus  be  analyzed  to  determine  the  screening  strength  of  T/C 
and  the  interaction  between  RV  and  T/C.  Since  RV  is  only  performed  once,  the 
RV  screening  strength  can  not  be  estimated  using  the  curve  fitting 
methodology. 

The  total  defect  distribution  is  provided  in  Figure  2.20  and  can  be  used  to 
estimate  the  T/C  screening  strength. 

The  'step*  at  5  cycles  is  due  to  the  effect  cf  RV  performed  between  cycles  5  and 
6.  The  resulting  T/C  distribution,  excluding  the  RV  effect,  is  shown  in  Figure 
2.20,  and  can  be  reasonably  modeled  by  the  simplified  equation  given  in  2.5.2. 

As  was  also  observed  for  Equipment  A,  the  removal  of  RV  defects  does  not 
result  in  a  distortion  of  the  T/C  defect  distribution  thus  indicating  that  RV  and 
T/C  defects  belong  to  different  populations: 

•  RV  does  not  precipitate  defects  subsequently  detected  by 
T/C. 

•  RV  does  not  remove  defects  that  could  be  removed  by  T/C, 

•  T/C  does  not  remove  defects  that  could  be  removed  by  RV. 

Equipment  C  data  for  a  different  time  frame  is  presented  in  Figure  2.21  and 
shows  similar  effects. 
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FIGURE  2.20  Equipment  C  Defect  Distribution.  1987. 
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d)  Comparison  of  Observed  and  Predicted  Screening  Strength  for  all 
Equipments.  The  screening  strengths  (actually  precipitation  effectiveness)  of 
temperature  cycling  and  random  vibration  determined  above  ai  e  compared  in 
Tables  2.5  and  2.6  to  the  HD6K  344  predicted  values  to  examine  the  accuracy 
of  the  HDBK.  As  illustrated  in  these  tables,  the  actual  results  compare 
favourably  with  the  prediction.  The  precipitation  efficiencies  for  temperature 
cycling  are  compared  in  Table  2.5.  Recall  precipitation  efficiency 
=  l-EXP(-Kt)).  The  average  of  the  precipitation  efficiency  across  ail 
equipments  was  .20.  The  average  temperature  transition  range  was  llO^C. 

The  comparative  HDBK  prediction  for  120®C,  4°C/min.  transition  rate  was  .19 
and  thus  agrees  with  the  observed  results. 

The  precipitation  efficiencies  for  random  vibr .  tion  are  compared  in  Table  2.6. 
Only  Equipment  A  test  flow  allowed  precipitation  efficiency  to  be  measured. 
The  average  value  of  over  2  years  was  .32  and  compared  favourably  with  the 
HDBK  estimate  of  .35. 

The  HDBK  equations  are  believed  to  be  useable  without  modification  (other 
than  the  change  in  interpretation  and  terminology  to  represent  precipitation 
effectiveness). 
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TABLE  2.5.  Comparison  of  Actual  and  •30D-HDBK-344  Predicted 
Screening  Strength  for  Temperature  Cycling. 


PROGRAM 

YEAR 

MEASURED 
K  FACTOR 
[BICYCLE-i) 

TEMPERATURE 

PER CYCLES 

AT[°C1 

SCREENING 
STRENGTH 
PER CYCLE 
(1  -e-K) 

Equipment  A 

1987 

.17 

.75 

122 

.20 

Equipment  A 

1988 

.24 

.75 

122 

.27 

Equipment  B 

1987 

.2 

1 

127 

.18® 

Equipment  B 

1988 

.25 

1 

127 

.22® 

Equipment  C 

1987 

.16 

1 

no 

.15 

Equipment  C 

1988 

.16 

1 

no 

.15 

AVERAGE  119  .20 

DOD-HDBK-344  .19 

[AT=120‘’C,4*C/MINJ 


TABLE  2.6.  Comparison  of  Actual  and  DOD-HDBK-344  Predicted 
Screening  Strength  for  Random  Vibration.  ~ 


PROGRAM 

YEAR 

K  FACTOR 
[CYCLE* ) 

GRMS 

SCREENING 
STRENGTH 
(5.5  GRMS,  5  MIN 

Equipment  A 

1987 

.08 

5.5 

.33 

Equipment  A 

1988 

.075 

5.5 

.31 

AVERAGE  .32 

DOD-HDBK-344  .35 

[6.5  grms,  5MIN] 
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2.5.3  Analysis  of  Assembly  level  screening  data.  Partof  the  economic 
optimization  process  of  ESS  requires  the  flexibility  of  performing  ESS  at 
earlier  or  more  cost  effective  stages  of  production.  It  was  thus  useful  to  analyze 
the  ESS  results  from  assembly  level  testing  to  evaluate  the  e^ectiveness  of 
ESS  at  that  stage  compared  with  to  system  level  ESS.  The  defect  and  removal 
data  for  assembly  and  system  level  ESS  was  provided  in  Table  2.4.  As 
discussed  in  2.4,  the  assembly  data  could  not  be  used  directly  since  it  contained 
a  mixture  of  defects, errors, and  removal  data;  thus,  the  latent  defects  were 
estimated  usin^  a  vtatistically  derived  factor.  An  approximate  value  for  SS 
was  calculated  uv  dividing  the  assembly  level  defects  by  the  sum  of  the 
assembly  and  system  level  defects. 

The  results  of  this  are  shown  in  Table  2.7  and  are  of  particular  concern  because 
of  the  low  value  of  assembly  level  ESS.  The  data  indicates  that  lower  level 
ESS  is  not  as  effective  as  system  level.  This  observation  is  further  supported  by 
the  existence  of  part  problems  that  theoretically  should  have  been  eliminated 
by  part  rescreening  at  incoming. 

In  fact,  the  data  for  Equipment  A,  B  and  C  are  similar  at  assembly  level  even 
though  Equipment  A  has  parts  rescreening  and  assembly  ESS,  Equipment  B 
has  assembly  level  ESS,  and  F-18  has  neither  (i.e.  no  lower  level  ESS). 

There  are  3  possible  reasons  for  this: 

i)  defects  are  not  precipitated  by  assembly  level  ESS. 

ii)  the  defects  are  introduced  subsequent  to  the  screen. 

iii)  defects  are  precipitated  but  not  detected 

The  possibility  that  defects  are  not  precipitated  is  not  logically  supported  by 
the  existence  of  TC  defects  at  system  level  and  the  correlation  between  actual 
and  predicted  precipitation  efficiency  at  system  level.  To  examine  the  premise 
that  the  defects  are  introduced  subsequent  to  assembly  level  ESS,  the  data  in 
Table  2.8  needs  to  be  considered. 
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TABLE  2.7.  ACTUAL  SYSTEM  AND  ASSEMBLY  LEVEL  DEFECT 
SUMMARY. 


TOTAL 

ASSEMBLY  LEVEL 

ESTIMATED 

TOTAL 

ASSY 

DEFECTS 

ASSEMBLY 

SYSTEM 

LEVEL 

AND 

LEVEL 

LEVEL 

SCREENING 

ERRORS® 

DEFECTS  PER  SYS 

DEFECTS 

STRENGTH® 

(DEF/SVS) 

(.23  X  TOT)® 

PER  SYS 

Equipment  A  2.64 

.66 

1.44 

.31 

Equipment  B,  MDI®  5.97 

1.49 

2.45 

.38 

Equipment  B,  MDRI®4.21 

1.05 

1.18 

.47 

Equipment  B,  HUD®  2.26 

.57 

1.52 

.27 

Equipment  C  3.59 

.90 

2.54 

.26 

®  ASSY  =  CARD  AND  MODULE  LEVEL  TESTING 
®  FACTOR  .25  BASED  UPON  REVIEW  OF  EQUIPMENT  A  DATA 
®  SS  =  (DEF  AT  ASSY)/(  DEF  AT  ASSY  +  DEF  AT  SYS) 

®  EQUIPMENT  B  PRODUCTION  DOES  NOT  USE  ESS  AT  THE 
ASSEMBLY  LEVEL 


TABLE  2.8.  Eouipment  A  System  Level  Defect  Distribution. 


FAULT  TYPE 

TOTAL 

DEFECTS 

TOT 

ENVIRON¬ 

MENTAL 

DEFECTS 

R/V 

DEFECTS 

T/C 

DEFECTS 

TOTAL 

1.44 

.968 

.207 

.766 

PARTS 

.99 

.709 

.154  , 

.555 

WORKMANSHIP 

.45, 

.259 

.048 

.211 

From  this  table,  it  can  be  seen  that  the  TC  defect  population  (.766  at  system 
level)  is  dominated  by  part  failures  (.555)  and  thus  existed  (as  latent  or  patent 
defects)  before  assembly  ESS.  The  lower  assembly  level  screening  strength  is 
thus  not  explainable  by  defects  being  introduced  subsequently.  The  logical 
conclusion  is  that  defects  are  precipitated  but  are  not  detected  until  system 
level  testing. 

This  conclusion  is  supported  by  the  handbook  that  suggests  that  50%  to  80%  of 
precipitated  defects  require  stress  concurrent  with  testing  to  be  detectable. 
Since  assembly  ESS  is  performed  without  testing  during  the  stress  application, 
the  ’observed’  screening  strength  would  be  between  20%  and  50%  as  limited  by 
detection  efficiency. 

These  results  indicate  that  the  effectiveness  of  assembly  level  screening 
strength  is  dominated  by  the  detection  efficiency  term  and  is  relatively 
ineffective  because  of  the  absence  of  a  stress  concurrent  with  the  test.  A  stress 
concurrency  factor  was  thus  recommended  and  included  in  the  revised  HDBK 
recommendations  for  estimating  detection  efficiency. 


Another  interesting  observation  is  that  if  the  thermal  shock/cycling  associated 
with  the  assembly  processing  eg  soldering  is  considered  and  modeled  then 
provided  SS  is  detection  efficiency  dominated  the  effective  assembly  level  ESS 
of  systems  with  temperature  cycling  eg  Equipment  A  and  Equipment  C  is 
nearly  the  same  as  for  those  with  no  temperature  cycling  eg  MDI,MDRI,HUD. 
This  would  explain  the  high  assembly  level  fallout  on  Equipment  B  as 
observed  in  Table  2.8. 
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The  methodology  proposed  for  the  handbook  addresses  these  considerations; 
hence,  the  ESS  optimization  considerations  and  pi  ocedures  remain  valid. 

2.6  Analysis  of  HDBK  cost  effectiveness  procedures.  The  primary  objectives 
of  ESS  are  to  precipitate  defects  such  that  their  cause  can  be  identified  and 
eliminated  (or  controlled)  and  thus  defect  density  reduced  and,  recognizing 
that  there  is  a  field  (MTBF)  requirement  that  is  considered  satisfactory  and 
thus  some  level  of  defect  density  that  can  be  tolerated,  precipitate  and  remove 
sufficient  defects  such  that  the  acceptable  defect  level  would  not  be  exceeded. 
The  ESS  program  thus  i)  determines  the  amount  of  screening  (strength) 
required  to  achieve  field  requirements  foi'  a  given  system  complexity  and 
defect  density  ii)  determine  the  screening  t3rpe  and  placement  to  optimize  the 
factory  screening  costs  and  iii)  ensure  the  process  is  under  control. 

This  approach  provides  the  customer/fleld  with  equipment  that  meets 
reliability  requirements  and  a  methodology  for  the  producer  to  minimize  his 
production/screening  costs  and  thus  customer  procurement  cost.  Since  defect 
reduction  improves  both  reliability  and  production  cost,  the  optimum  tends 
towards  defect  prevention.  However,  it  is  unlikely  that  defects  can  be  entirely 
eliminated;  thus,  defect  removal  using  ESS  remain^  a  requirement  to  some 
degree.  The  cost  effectiveness  of  an  ESS  program  is  concerned  with  the  cost  of 
meeting  a  customer’s  reliability  requirement  and  must  consider  both  the 
producer’s  cost  of  screening  a  defect  and  the  cost  of  a  field  failure  to  the  user 
and  producer. 

The  HDBK  currently  addresses  the  combined  user  producer  cost  by  assuming 
the  cost  of  a  defect  in  the  field  to  be  $1000.  This  value  then  becomes  a 
"threshold”  cost  and  has  the  implication  that  if  the  cost  of  factory  ESS  exceeds 
$1000  per  defect,  it  may  be  more  economical  to  reduce  factory  ESS  and  let  the 
defects  occur  in  the  field  for  non-criticai  systems.  However,  for  mission  critical 
high  reliability  systems,  cost  wovild  be  of  secondary  concern. 
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Consistent  with  this  intent,  the  HDBK  procedures  were  modified  such  that  if 
reliability  requirements  can  not  be  achieved  the  ESS  plan  would  be  required  to 
provide  the  projected  marginal  cost  per  defect  i.e.  the  factory  ESS  cost  of 
removing  the  excess  defect  ^er  system.  The  procuring  activity  could  then 
determine  whether  this  value  is  reasonable  compared  to  the  cost  of  a  defect  in 
the  field  and  whether  reliability  requirements  pould  be  relaxed  for  certain  cost 
considerations.  The  $1000  threshold  provided  in  the  HDBK  is  a  reference 
guide  only  and  must  established  by  the  customer.  In  the  absence  of  a  known 
threshold  cost,  the  optimization  procedures  should  assume  that  the  desired 
reliability  must  be  achieved  and  thus  provide  the  user  with  the  methodology 
for  designing  or  modifying  the  screening  program  to  minimize  screening  cost 
and/or  to  optimize  the  cost  of  achieving  the  required  field  MTBF.  Although  the 
amount  of  screening  required  by  the  producer  is  determined  by  the  field 
reliability  requirement  and  defect  density-complexity  of  the  equipment,  Uie 
producer  has  flexibility  in  what  screen  to  use,  the  stress  level,  and  the  position 
of  the  screen  in  the  test  flow  and  has  constraints  on  the  assembly-test  sequence 
due  to  calibration  and  performance  verification  testing  and  constraints  on 
permissible  stress  levels  imposed  by  the  equipment  design.  The  cost  analysis 
procedure  assists  the  producer’s  optimization  of  these  trade  offs. 

The  methodology  of  Procedure  D  identifies  Bxed  costs  which  are  actually  the 
non  recurring  costs  to  establish  the  screening  capability  (for  a  given  program) 
and  variable  costs,  which  are  the  recurring  costs  for  performing  the  screens. 

The  principle  factors  considered  in  the  cost  analysis  and  identified  in 
,  Procedure  D  are: 

i)  cost  of  testing 

ii)  cost  of  environmental  stressing 

iii)  cost  of  repair/rework  and  associated  analyses 

iv)  cost  of  repeating  stresses  and  tests,  depending  upon  the  test 

flow/failure  free  requirements 
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It  should  be  noted  that  the  HDBK  approach  optimizes  the  cost  based  on  the 
program  cost  by  combining  the  non  recurring  cost  with  the  recurring  cost  per 
system  multiplied  by  the  total  systems  to  be  produced.  An  alternate  approach 
is  to  amortize  the  non  recurring  costs  based  upon  ESS  equipment  useful  life 
and  utilization,  thereby  converting  all  costs  to  a  per  system  bases.  This  later 
approach  is  useful  when  the  equipment  can  be  shared  among  several  programs 
and/or  if  the  programs  have  an  indeterminate  total  quantity. 

The  cost  optimization  methodology  was  applied  and  validated  by  the  study, 
however,  cost  results  are  company  confidential  and  thus  not  provided  in  the 
report.  The  methodology  however  is  illustrated  in  5.0. 

2.7  Analysis  of  HDBK  Failure  Free  Acceptance  Testing  (FFAT).  The  present 
HDBK  contains  a  requirement  for  a  Failure  f'ree  Acceptance  Test  (FFAT)  as 
described  in  HDBK  para.  5.4,  Procedure  C.  The  mathematical  derivation  is 
provided  in  Appendix  C  of  the  HDBK. 

The  validation  and  analysis  performed  dn  the  HDBK’s  FFAT  concerned  i)  the 
validation  of  the  mathematical  model  and  procedure,  ii)  validation  of  the 
assumptions  used  for  the  model  and  iii)  a  review  of  the  implications  of  FFAT 
in  general.  A  computer  program  was  specifically  developed  to  analyze  the 
FFAT  mathematics  and  procedures. 

The  FFAT  derivation  given  in  Appendix  C  of  the  HDBK  was  reviewed  and 
validated  with  the  exception  of  a  typographical  error  in  equation  C-4  (i.e., 
should  be  exp[-D  exp  -(Aot)]  and  concerns  related  to  the  t  '.  'del  assumptions 
that  are  discussed  below. 

,As  an  illustration,  the  Act  -  Ai>'\o  tables  in  the  handbook  (ref.  Tables  5.19  - 
5.20)  were  recreated  using  the  developed  software  and  Figure  2.22  herein  can 
be  seen  to  correlate  with  HDBK  Table  5.19  and  thus  validates  the  handbook 
mathematics. 
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Note  lambda  is  represented  by  the  letter  L  in  the  above  tables 
FIGURE  2.22,  Validation  of  DOD-HDBK-344  Table  5.19. 
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Although  the  mathematics  described  by  HDBK  Appendix  C  were  applied  to  an 
FFAT,  problems  were  uncovered  with  the  details  of  Procedure  C  of  the  HDBK. 
Following  from  Appendix  C,  the  FFAT  is  determined  based  upon  the  amount  of 
’testing  required  to  ensure  that  the  MTBF  at  the  completion  of  testing  meets 
system  requirements  with  some  specified  confidence  level. 

The  specified  parameters  are  thus  i)  the  required  system  Xs  the  desired 
confidence  level,  and  iii)  the  inherent  Xq. 

The  user  has  only  one  degree  of  freedom  and  can  thus  select  only  one  of  Xq 
(failure  rate  for  defect).  Dins  (initial  defects  at  start  of  FFAT) ,  or  T  (duration 
of  FFAT).  Having  selected  one  of  these  parameters,  the  other  two  are 
determined  by  solving  the  equations  in  Appendix  C.  If  the  user  selects  a  Xd 
(i.e.  this  is  equivalent  to  specifying  a  screening  stress  level),  then  the  HDBK 
Tables  5.19  -  5.28  can  be  used  to  solve  for  T  and  hence  Din3-  These  calculations 
and  procedures  are  performed  by  the  LSL  software. 

This  is  illustrated  in  Figure  2.23  where  the  analysis  software  was  used  to 
determine  the  FFAT  for  a  system  with  a  required  Xs  of  .001,  an  inherent  Xq  of 
.000909,  a  confidence  level  of  90%,  and  a  selected  Xoof  .01.  For  this  example. 
Din  max  was  found  to  be  1.82  and  would  give  a  first  pass  factory  yield  of  10% 
(i.e.,  confirming  the  90%  confidence  limit). 
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Specify 


CAleulAta 


Sy*taa  failor*  r«ta  0.0010 

Lo  faiiiir*  rata  0.0009 

Confidanca  laval  for  FFAT 


thus  MTBF* 
thtta  MIBF« 


Tba  syataa  failura  rata  dua  to  dafacts  ia  thus: 


Ld.OETout  ■ 

Strata  laval  /  Ld  > 


Caleulata 


Datamlna 


Caleulata 


0E7  eat«  0.0091 

Tiald  (*axp(>0E7dut) )  99.10%  {pareat 

in  fii 

U/Lo-  11 

a 

Id.T  ■  5.3  {uaa  t< 

a 

TEST  HRS-  530 

Starting  daf  ■  1.8212 

fEAT  Seraanlng  Stran;^  0.995 

Probability  of  paaaing  failura  fraa- 
ia  ahoold  ••  (1-  apaeifiad  conf.) 

Firat  paaa  "FFAT''  yiald  va  dafacta/ayatan: 


C.OOOl  MTBF  11000 

0.01  MTBF  100 

0.0091 

99.10%  {pareaat  dafaet  fraa  ayataaa 
in  fiald  aftar  ESS  t  FFAT} 


5.3  {uaa  tablaa  or  itarativa  aolntinn 

530 

1.8212 

0.995 


10.09% 


F 

lat  paaa  yld 

0 

61.77% 

0.001 

61.70% 

0.01 

61.15% 

0.1 

55.92% 

0.25 

48.16% 

0.5 

37.56% 

0.75 

29.29% 

1 

22.84% 

1.25 

•  17.81% 

1.5 

13.89% 

1.75 

10.83% 

2 

8.44% 

2.5 

5.13% - 

3 

.  3.13%  • 

FIGURE  2.23.  Failure-Free  Acceptance  Test  ’FFAT  Example 
Note  lambda  is  represented  by  the  letter  L  in  the  above  tables. 
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The  problem  associated  with  the  procedural  details  is  that  steps  2,  5,  and  7 
imply  that  there  is  more  than  one  degree  of  freedom  and  that  the  user  can 
arbiU  nly  select  or  determine  Din3,  TS  and  SS.  This  is  not  believed  to  be 
posuV '  r  ;.:!d  thus  Procedure  C  would  need  to  be  rewritten.  Before  considering 
re  »  riti  Procedure  C,  however,  the  assumptions  and  rationale  of  an  FFAT 
nee».  ‘  i  b  re  examined. 

The  de.-ivation  of  the  FFAT  is  based  upon  Appendix  C  equation  C-2. 

1  -  CONF  =  exp  [  -XoT  -D  (1  -  exp  (-XdT)] 

The  problem  with  applying  this  equation  is  that  Xq  and  Xd  relate  to  different 
stress  levels.  If  the  user  selects  a  higher  factory  stress  level,  for  example,  a  Xq 
for  5  g  RMS  vibration  as  opposed  to  1  g  RMS,  then  Xq  should  also  increase  since 
both  are  a  function  of  stress  level  (s).  The  Xd  term  inherently  addresses  the 
relationship  with  stress;  however,  the  stress  dependency  is  missing  for  Xq. 

In  this  sense,  the  equation  is  an  "apples  and  oranges”  mix;  hence,  its  solution  is 
not  necessarily  valid.  When  applying  the  FFAT  procedure  the  user  can  not 
arbitrarily  select  Xd  without  affecting  Xq  and  the  Procedure  is  thus  believed  to 
be  flawed. 

From  the  foregoing,  the  FFAT  seems  to  be  flawed  from  theoretical  and 
procedural  aspects.  From  a  philosophical  aspect  it  is  also  undesirable. 

The  concept  of  an  FFAT  is  contrary  to  the  purpose  of  ESS  and  the  HDBK  in 
that  the  FFAT  determines  the  minimum  ESS  program  and  prevents  the  user 
fnim  optimizing  or  reducing  the  screening  program  to  be  less  than  the  FFAT 
minimum.  This  can  remove  the  incentiye  for  the  user  to  reduce  defects  since 
the  possibility  of  a  defect  density  -  screening  strength  trade  off  is  eliminated  by 
the  imposed  minimum  screening  requirement  of  the  FFAT. 

These  points  are  illustrated  in  the  example  provided  in  Figure  2.22.  For  the 
desired  90%  confidence  FFAT,  the  XqT  is  required  to  be  5.3  (i.e.,  530  hrs 
screening  at  Xq  =  .01),  the  required  screening  strength  [  i.e.,  1  -  exp  (  -XqT)]  is 
thus  0.995  regardless  of  the  initial  defect  density.  If  the  user  were  to  reduce 
the  defect  density  to  0  through  effective  corrective  action,  the  FFAT  would  be 
unchanged  and,  even  with  zero  latent  defects,  the  producer  would  suffer  a 
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FFAT  yield  of  62%  (due  to  the  limiting  failure  rate  Xq)*  In  this  example,  a  zero 
(screenable)  defect  system  would  still  require  530  hrsi  at  a  yield  of  62% 
Resubjecting  the  "failed”  system  to  the  FFAT  would  incur  a  subsequent  62% 
yield  ^d  the  user  would  becomes  caught  on  a  costly  vicious  circle  and  the 
average  screening  time  per  system  would  approach  680  hours.  The  extended 
screening  time  would  shorten  the  useful  life  of  the  equipment  and  could  be 
sufHcient  to  induce  fatigue  type  failures  soon  after  shipment  and  would  also 
adversely  affect  the  cost  of  the  equipment  due  to  the  unnecessary  test  and 
rework  costs  (for  \q  failures). 

The  FFAT  is  structured  such  that  if  the  producer  achieves  the  required  As, 
then  his  first  pass  FFAT  yield  is  1-  CONFIDENCE,  i.e.,  10%  for  a  90% 
confidence  FFAT  and  he  is  thus  severely  punished  for  meeting  requirements. 
Because  of  the  low  discrimination  ratio  for  an  'accept  on  zero’  type  plan,  the 
FFAT  confidence  levels  need  to  be  carefully  considered  and  set  and  may  need 
to  be  less  than  90%. 

Since  a  lower  confidence  level  may  be  viewed  as  unsatisfactory  by  the 
customer;  an  alternative  means  of  demonstrating  reliability  other  than  FFAT 
is  required.  The  recommended  approach  is  to  use  SPC  and  the  available  data 
from  the  HDBK  344  ESS  Program.  In  essence,  the  purpose  of  the  FFAT  is  to 
provide  confidence  that  the  systems  are  sufficiently  defect  free.  This 
confidence  however  can  be  obtained  from  the  SPC  of  the  ESS  fallout  without 
the  cost  of  redundant  and  perhaps  unnecessary  testing. 

The  preferred  approach  for  of  ESS  would  be  that,  provided  sufficient  corrective 
action  can  not  be  taken  to  prevent  defects,  all  systems  undergo  similar 
screening  that  has  a  sufficient  strength  to  ensure  that  field  reliability 
requirements  can  be  achieved.  The  emphasis  is  thus  on  defect  prevention  with 
ESS  being  a  "short  term”  solution.  During  ESS,  it  is  useful  to  expose  all 
systems  to  the  same  amount  of  screening  in  order  to  have  a  more  controlled 
fatigue  life  distribution  thus  permitting  regularly  scheduled,  preventive 
maintenance  to  be  used  to  maximize  the  gains  in  MTBF  afforded  through 
defect  prevention  and  ESS. 

2.8  Analysis  of  HDBK  Statistical  Process  Control  Methodology.  If  the  ESS 
program  is  to  be  controlled  and  used  to  indicate  compliance  with  reliability 
requirements,  some  form  of  monitoring  is  required.  The  HDBK  recommends 


the  use  of  Statistical  Process  Control  (SPC)  and  PARETO  charts  for  this 
purpose.  This  methodology  and  recommended  changes  are  discussed  in  the 
folloudag. 

a)  DOD-HDBK-344  Procedure  E  requires  SPC  charts  to  be  prepared  for 
these  ESS  parameters: 

i)  INCOMING  DEFECTS 

ii)  FALLOUT 

iii)  OUTGOING  DEFECT  DENSITY  TO  OUTGOING  DEFECTS 

These  charts  are  based  cn  a  Poisson  distribution,  i.e.,  mean  =  variance  =  p 
and  belong  to  a  class  of  SPC  chart  that  is  based  on  attributes  (i.e.,  C  chart) 
rather  than  variables  (  X  and  R  charts).  (Ref.  STATISTICAL  QUALITY 
CONTROL,  E.L.  GRANT,  and  R.S.  LEAVENWORTH,  MCGRAW  HILL]. 

The  methodology  described  in  the  handbook  suggests  SPC  monitoring  on  an 
individual  system  basis.  For  ESS  monitoring,  however,  it  would  be  preferable 
to  use  a  normalized  monitoring  scheme  based  on  the  average  defects  per 
system  ie  defect  density  .This  type  of  monitoring  chart  is  a  "U"  chart  where  u  = 
total  nonconformities/total  units  inspected.  The  control  limits  are  based  on 
u  +  -  3  V(  u/n)  where  n  is  the  sample  size. 

In  addition  to  being  more  directly  related  to  ESS  parameters  it  permits  the 
user  to  also  track  performance  on  a  time  basis,  eg.,  daily/weekly/monthly,  etc. 
and  since  n  can  be  made  large,  the  sensitivity  can  be  increased  (i.e..  Sigma  -* 
0). 

These  charts  in  essence  monitor  the  variation  in  the  mean  defect  density  and 
are  thus  a  test  for  homogeneity.  The  inherent  problem  is  that  TQM  and  ESS 
approaches  are  aimed  at  continuous  improvement;  thus,  the  process  mean 
should  be  continually  improving  and  not  remain  constant. 

A  suggested  modification  to  the  HDBK  and  SPC  approach  is  to  determine  the 
current  process  capability  using  regression  analysis  and  thus  attempt  to  detect 
and  quantify  the  improvement  rate.  The  expected  variation  due  to  limited 
sample  size  would  be  determined  using  this  value  for  the  process  capability 
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Tmean)  and  the  expressions  given  below.  This  variation  would  be  plotted  as 
control  limits  (using  the  formula  given  above)  and  used  to  identify  possible 
short  term  trends  and  problems. 

I 

When  applying  these  charts  for  small  sample  sizes,  the  quantization  effect  can 
become  significant  and  may  create  the  misconception  of  an  out  of  control 
process.  For  example  control  limits  of  u  ±  .01  can  not  be  resolved  with  a 
sample  size  of  10  since  the  actual  data  cpuld  only  be  plotted  with  a  resolution  of 
■  ±.1. 

Since  the  purpose  of  the  ESS  is  to  ensure  that  custo:.aer  reliability 
requirements  are  satisHed,  the  control  chart  must  reflect  th  .  ‘equirements  as 
well  as  the  current  process  capability.  It  is  not  adequate  to  merely  'control’  the 
process.  The  process  must  both  be  in  control  and  capable  of  achieving 
requirements.  (Aside:  The  definition  of  requirements  may  also  require  further 
clarification.  Presently  most  requirements  relate  to  a  mean  of  some 
performance  index,  eg.,  MTBF,  DPM,  etc.,  however  there  may  be  a  need  to  also 
specify  the  allowable  deviation,  eg.,  more  than  X%  of  the  systems  must  have 
less  than  Y  defects.) 

To  resolve  these  concerns  and  simplify  the  application,  LSL  has  developed  PC 
software  to  produce  a  modified  form  of  SPC  control  chart.  The  software 
accommodates  a  variable  sample/production  size  and  adjusts  the  'control’ 
(actually  statistically  expected  variations)  limits  accordingly.  The 
requirements  are  determined  directly  from  the  ESS  program  plan  and 
therefore  are  directly  related  to  required  field  reliability.  An  illustrative 
example  of  the  modified  SPC  chart  is  shown  in  Figure  2.24. 

b)  PARETO  Charts.  A  useful  chart  to  help  identify  the  cause  of  problems 
is  the  PARETO  chart.  The  PARETO  typically  examines  the  frequency  of 
various  causes  of  non  conformities.  A  concern  with  the  PARETO,  however,  is 
that  a  conventional  PARETO  identifies  the  most  frequent  cause  but  not 
necessarily  the  most  important  cause  and  can  over  look  what  is  expected  based 
upon  other  considerations,  for  example  complexity. 

To  overcome  this,  a  modified  PARETO  is  suggested  that  charts  not  only  actual 
results,  but  compares  them  with  the  expected  results  based  on  complexity  and 
statistical  significance. 
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A  PARETO  anaiysis  »o(lwar«  package  was  developed  by  LSL  for  this  purpose 
and  would  be  used  on  a  routine  basis  to  compare  actual  to  expected 
defV«rW system  fi.r  the  various  assembUes  and  subassemblies. 

An  illustrative  example  of  the  modified  PARETO  is  provided  in  Figure  2,25. 
This  modified  PARF-TO  compares  the  nctual  defects  per  assembly  to  the  values 
that  were  expected  based  on  assembly  complexity  and  detennined  during  the 
ESS  planning  phase.  To  allow  for  statistical  variation  due  to  sample  size,  the 
expected  values  are  indicated  as  1 sigma  bars  (assuming  a  Poisson 
distribution).  More  con  vent-onal  PARE!  0«  can  be  produced  if  desired. 

The  SPC  charts  on  Total  Defects.  Part  Defects  and  Workmanship  Defects,  etc., 
arc  useful  to  indicate  the  overail  status;  however,  the  PARETO  for  assemblies 
it  useful  to  identify  possible  design  weakness  or  faults.  Since  a  dcsig^n  fault  is 
often  speciPic  to  a  particular  assembly,  the  PARETO  identifies  those 
assemblies  with  defect  densities  above  expected  levels  (based  on  the  overall 
process  capability  for  parts  and  workmanship),  and  thereby  identifies  possible 
design  weaknesses  for  more  indepth  analysis. 

Conversely,  assemblies  with  abnormally  low  defects  may  indicate  test 
detection  efficiency  problems,  or  low  screening  stresses.  As  discussed  in  the 
current  HDBK,  the  user  must  be  vigilant  and  examine  all  abnormal  results  for 
both  'good'  and  'bad’  indicators. 

c)  Relating  Required  Reliability  to  Remaining  Defects.  The  monitoring 
and  control  procedure  described  in  the  HDBK  if  modified  as  suggested  herein 
can  be  applied  and  used  for  an  effective  control  program  provided  the  customer 
reliability  requirements  can  be  related  to  F-SS  parameters,  eg.  Outgoing 
Defect  Density. 

A  consistent  methodology  for  establishing  this  relationship  however,  does  not 
presently  exist  in  the  handbook.  A  related  problem  is  that  the  method  for 
determining  what  screening  strength  is  required  based  upon  the  plan  and/or 
the  actual  observed  defect  densities  also  does  not  exist. 
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The  procedure  should  be  able  to  determine  the  screening  strength  required  to 
satisfy  customer  reliability  requirements  assuming  the  HDBK  estimated 
defect  densities  ar^  being  achieved.  If  however,  the  requirements  established 
in  the  plan  are  not  being  achieved,  the  user  also  needs  a  methodology  for 
determining  how  much  additional  screening  strength  is  required,  until  the 
process  is  back  in  control  and/or  in  line  with  the  plan. 

There  are  two  methods  available  to  the  user  for  translating  reliability 
requirements  into  a  factory  requirement  for  remaining  defects  (per  system). 
The  first  method  is  to  model  the  field  operating  environment  in  terms  of  actual 
stress  levels  and  conditions  eg  temperature  cycles  with  temperature  range  and 
transition  rate  and  vibration  with  G  level  (power  spectral  density)  and 
duration.  Knowing  these  conditions,  the  user  can  then  directly  apply  the 
precipitation  efTiciency  equations  given  in  the  HDBK.  Since  the  detection 
efficiency  of  the  field  is  1  (by  definition)  the  screening  strength  is  equal  to  the 
precipitation  efficiency  and  is  thus  known  for  the  corresponding  duration  in 
the  field.  The  defects  occurring  over  this  field  operating  interval  are  then 
determined  as  the  product  of  the  remaining  defects  (at  the  conclusion  of  factory 
ESS)  times  the  screening  strength  of  the  field. 

The  maximum  allowable  defects  remaining  at  the  completion  of  factory  ESS 
can  thus  be  expressed  in  terms  of  the  maximum  allowable  field  defects  as 
follows: 

Maximum  Defects  (field  stress)  =  Maximum  allowable  field  defects  /  Field 
Screening  Strength. 

The  maximum  allowable  field  defects  can  be  determined  by  multiplying  the 
required  MTBF  by  the  operating  interval.  This  expression  determines  factory 
defects  relative  to  the  operational  stress  levels.  To  determine  the  defects 
relative  to  the  factory  ESS  stress  level,  this  value  would  be  multiplied  by  the 
appropriate  stress  adjustment  factor. 


The  second  method  would  be  used  if  the  actual  Held  stress  conditions  are  not 
known  or  available  and  uses  equation  2.2  with  an  appropriate  value  for  K.  The 
(l*e-kt)  term  in  this  equation  is  the  precipitation  effectiveness  of  the  field  and 
thus  determines  the  screening  strength  as  discussed  above.  The  remaining 
calculations  would  be  performed  as  described  for  method  1  above.  The  value  of 
K  can  be  determined  from  experience  data  and  or  published  literature  and  is 
probably  in  the  order  of  1/300  to  1/1000  hours  depending  on  operating 
environment. 

In  summary,  a  field  requirement  fo»  MTBF  or  FFOP  can  be  translated  into  a 
factory  requirement  for  the  maximum  outgoing  defects  per  system.  This  is 
accomplished  by  using  either  of  the  methods  outlined  above  to  estimate  the 
equivalent  screening  strength  of  a  specified  interval  in  the  field.  This  estimate 
of  field  screening  strength  should  be  refined  as  actual  data  becomes  available. 
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PARTC 

Introduction.  The  etudy  program  required  the  ESS  methodology  of  DOD- 
HDBK-344  to  be  applied  under  actual  factory  conditions  and  the  results  used 
to  determine  and  correct  deficiencies  in  the  procedures.  Part  B  described  the 
application  of  the  HDBK,  the  analysts  of  the  observed  results,  and  various 
recommendations  for  changing  the  handbook.  This  part  provides  a  procedure 
by  Procedure  critique  of  the  HDBK  and  describes  specific  problems  and  the 
recommended  changes.  Part  C  thereby  relates  test  the  results  of  Part  B  back 
to  the  HDBK  and  explains  the  reasons  for  revised  handbook  provided  as  Part 
D. 

2.  Procedure  A1  Incoming  Defect  Density  Requirement. 

Purpose.  To  provide  assurance  that  the  part  fraction  defective  is  below 
R&M  2000  policy  guideline,  i.e.,  100  DPM  (FY90) 

b)  Comments. 

i)  A  current  industry  interpretation  is  that  the  100  DPM 
requirement  is  applicable  to  electrical  test,  i.e. ,  patent  errors. 
It  is  important  to  recognize  that  latent  defects  determine 
reliability  and  should  be  the  concern  of  HDBK  344.  The 
Electronic  industry  must  address  latent  defects  as  well  as 
patent  errors. 

ii)  The  significance  of  the  100  DPM  level  needs  to  be  determined 
based  upon  specific  reliability  requirements.  The  proposed 
HDBK  provides  the  methodology  for  determining  the 
appropriate  level,  whether  it  be  greater  or  less  than  100  DPM. 
Ileferencing  the  100  DPM  guideline  in  the  HDBK  is  thus 
unnecessary  and  potentially  misleading. 

iii)  EIA  Procedure  554-88,  may  be  useful  for  meEisuring  outgoing 
non  conforming  levels  in  DPM. 
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3.  Procedure  A2.  Estimating  Incoming  Defect  Dgn$itv.  The  reconunendation 
U  to  eliminate  the  direct  reference  to  the  100  DPM  guideline. 

a)  Purpoae.  To  eatimate  the  incoming  defect  denaity  (Din) 

b)  Comments. 

i)  The  procedure  eatimatea  factory  incoming  defect  levels  based 
on  the  operating  environment.  This  approach  is  not 
applicable  when  estimating  factory  ESS  fallout  since  factory 
stress  levels  do  not  directly  relate  to  the  operational 
environment.  The  recommendation  is  to  create  a  defect 
density  data  base  that  is  applicable  to  a  reference  or  base  line 
stress  level  equivalent  to  R&M  2000.  Applicable  defect 
densities  at  other  stress  levels  would  be  obtained  using  an 
appropriate  Stress  Adjustment  Factor  /SAF. 

ii)  The  procedure  assumes  Din  increases  with  stress,  i.e.,  is  a 
function  of  the  operating  environment  Since  factory  ESS 
stress  levels  must  be  higher  than  the  application,  the 
implication  is  that  defects  screened  may  be  relevant  or  non 
relevant  The  basic  HDBK  model  is  thus  not  adequate  and 
stress  effects  must  be  included. 

iii)  Din  is  expected  to  vary  with  time  and  product  maturity  (ref 
HDBK  4.10.3.3.2  and  the  study  results).  The  direct 
application  of  HDBK  tables  thus  has  questionable  accuracy, 
and  the  user  must  be  provided  with  a  methodology  to  rescale 
the  tables  based  on  experience. 

iv)  the  level  of  accuracy  that  is  required  must  be  kept  in 
perspective.  The  tables  of  defect  density  should  be  used  for 
initial  planning  purposes  only  (ref.  HDBK  4.10.2.3.1) 

a)  Defect  densities  for  parts  and  workmanship  (hence 
goals  and  ESS)  must  be  adjusted  based  on  users 
actual  data. 
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b)  ibt  procfdur*  should  outlint  tht  method  of 
MtimAting  Din  and  SS,  etc.  from  factory  fallout  data 

c)  the  proc<Hiurt  should  emphasize  maximum  use  of 

user  data 

v)  The  perception  of  defects  being  either  part  or  workmanship 
may  be  affected  by  the  degree  of  root  cause  analysis  and  the 
elassincation  guidelines.  Without  detailed  analyses,  many  of 
the  problems  due  to  design  or  workmanship  may  be 
incorrectly  perceived  as  part  problems.  The  comparison  of 
defect  density  data  among  different  users  must  thus  be  made 
with  caution.  It  is  noteworthy  that  the  LSL  results  did 
correlate  with  the  HDBK  guidelines  ref.  4.10.2.3.3  on  the 
fisiction  of  parts  and  workmanship  defects  (after  the  LSL 
recommended  changes,  e.g.,  using  MIL-STD-2000  for 


complexity). 

. 

garts 

Workmanshio 

LSL 

17%  -  33% 

HDBK  344 

60% -70% 

20% -30% 

(ref.  4.10.2.3.3) 

vi)  The  HDBK  acknowledges  the  need  to  proportion  Din  as  RV 
and  T/C  defects  (ref.  HDBK  5.2.3. 1).  This  is  necessary  to 
achieve  the  correct  balance  of  RV  and  T/C  screens.  The  LSL 
results  indicate  that  RV  and  T/C  defects  tend  to  belong  to 
different  populations  and  thus  it  may  be  unrealistic  to  expect 
T/C  to  remove  RV  defects  and  vice  versa.  The  detailed 
procedures  in  the  HDBK  need  to  be  changed  to  emphasize  the 
need  to  proportion  defects  into  RV  and  TC  defect  populations. 

The  LSL  results  correlated  with  HDBK  results  concerning  the 
proportion  of  RV  and  T/C  defects. 
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RV 

T/C 

LSL 

26% 

74% 

HDBK  344 

20% 

80% 

3.1  Procedures.  Determining  Screening  Strength. 

Purpose  .To  determine  the  Test  Strength’ TS  *  SS.DE  (Note  with  the 
recommended  changes  in  terminology  this  becomes  SS  s  P£  x  DE 
where  SS  is  screening  strength,  where  PE  is  Precipitation  Efficiency 
and  DE  is  Detection  Efficiency .i 

b)  Comments. 

i)  The  significance  of  testing  concurrently  with  stress  application 
should  be  emphasized.  For  example,  if  the  assembly  level  test 
coverage  is  85  •  99%  (ref.  HDBK  table  4.6)  and  SO  •  80%  of  defects 
require  stress  for  detectability  (ref.  HDBK  4.10.3.4.2)  then  the 
easembly  level  SS  is  low  (.17  -  .5).  *rhis  low  value  of  TS  (i.e.  now  SS) 
was  observed  by  the  study  during  assembly  level  screening  (i.e., 
<.3). 

iii)  The  limited  DE  implies  that  undetected  patent  (precipitated  latent) 
defects  will  escape  to  the  next  stage.  This  can  create  an  apparent 
'on  receipt’  or  'infant  mortality’  problem.  The  analysis  procedures 
must  thus  recognize  escaping  patent  defects. 

'  iii)  The  estimate  for  RV  precipitation  effectiveness  must  be  modified 
to  recognize  an  axis  sensitivity  (ref.  HDBK  4.10.3.3.3.C).  The  study 
revealed  a  strong  axis  sensitivity  factor  i.e.  perpendicular  to  the 
plane  of  the  PCB’s.  The  complexity  vectors  should  be  computed  for 
the  X,  y,  and  z  axes  and  the  calculation  of  PE  for  RV  adjusted  in 
proportion. 

iv)  It  is  important  to  emphasize  that  the  stress  parameters  (t,  dT/dt, 
grms,  etc.)  should  apply  to  the  unit  under  test  (UUT)  not  the 
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chamber.  The  calculation  and  modelling  must  thus  include  stres4 
transmission  characteristics  (ref.  HDBK  4.10.3.5)  and  recognize 
that  g  levels  may  be  enhanced  or  suppressed  and  dT/dt  is  limited  by 
thermal  mass  and  conductivity. 

Further,  stress  levels  vary  over  the  equipment/assembly 
ocmpUcating  identification  of  the  appropriate  stress  parameters 
and  thus  application  of  the  tables.  Thermal  and  vibrational  surveys 
on  the  simulated  or  completed  hardware  may  be  required. 

v)  The  study  validated  the  PE  tables  for  RV  and  T/C  (for  the  particular 
stress  levels  used).  However,  the  stress  levels  cited  (especially  for 
RV)  may  need  to  be  rescaled  to  be  consistent  with  iv.  The  stress 
levels  used  for  the  tables  should  be  the  actual  stress  levels  to  which 
the  equipment  is  subjected.  Because  of  equipu  ent  transmissibility, 
these  values  are  different  than  the  stresses  measured  at  the  output 
of  the  ESS  chambers  and  shakers.  Because  the  HDBK  tables  and 
equations  were  derived  using  data  from  actual  equipment,  the 
stress  levels  given  in  the  tables  imply  ESS  output  levels  but 
actually  represent  the  levels  seen  by  the  equipment  under  test  . 
These  levels  should  be  multiplied  by  the  transmissibility  of  the 
equipment  used  for  the  data  collection  In  order  to  make  the  tables 
more  generic. 

vi)  The  HDBK  mathematical  model  can  result  in  significant  errors  if 
incorrectly  applied.  Using  the  current  HDBK  terminology,  the 
model  is  based  on  TS  =  SS  .  DE.  Thus,  TScum  is  given  by  1  -  n  \ 
(1  -  TSj)(ref.  40.7  eq.  A-10)  the  resulting  TScum  is  thus  asymptotic 
to  1.  However,  although  the  SScum  is  asymptotic  to  1,  TScum  must 
be  asymptotic  to  the  detection  eBiciency  (DE).  Provided  similar 
tests  are  not  repeated,  the  error  is  probably  not  significant; 
however,  successive  tests  of  a  similar  nature  could  result  in 
significant  modelling  errors. 

3.4  Procedure  C  -  Failure  Free  Acceptance  Test. 

a)  Purpose.  To  provide  a  level  of  assurance  that  screening  is  (sufficiently) 
complete  (ref.  HDBK  4.10.3.8) 
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b  Comrognti. 

{)  Th*  procedur*  uset  extended  testing  et  high  stress  to  provide  • 
ttatisticnl  confldence  level  on  the  remaining  defects.  The 
procedure  has  no  consideration  of  prior  ESS  results  and  requires  a 
fixed,  demonstration  ESS  be  performed  (or  repeated)  regardless  of 
defect  levels  and  SPC  performance.  For  this  reason  FFAT  is  not 
considered  to  be  economically  prudent 

ii)  The  mathematical  basis  for  the  FFAT  (Appendix  C  equation  C-2) 
involves  mixed  stress  levels)  i.e.  Xo  is  at  the  operating  stress  and 
Xa  xt  ESS  stress  and  the  accuracy  is  thus  questionable. 

iii)  The  procedure  cannot  be  used  as  written  since  the  user  has  only  1 
degree  of  freedom  in  selecting  stress  levels  or  duration  and  would 
need  to  be  modified. 

iv)  Imposing  an  FFAT  can  impact  and  penalize  the  user  for 
achieving  the  required  reliability. 

If  its  user  satisfies  the  required  reliability,  the  first  pass  yield 
would  be  1  minus  the  confidence  level,  eg.,  if  the  confidence  level 
is  90%  the  factory  yield  would  be  10%  and  the  producer  would  be 
required  to  rework  90%  of 'good'  systems.  The  low  yield  would  be 
caused  by  failures  due  to  the  design  limit  occurring  at  a  constant 
failure  rate  (CF)  and  thus,  could  not  be  improved  without 
redesign.  Extended  test  times  are  required,  even  if  Diatent  is 
zero.  The  unnecessary  testing  and  rework  increase  cost  and 
potentially  degrade  reliability. 

▼)  The  FFAT  may  dominate  over  the  HDBK  344 
Quantitative/Adaptive  approach  -  i.e.,the  minimum  ESS  testing 
imposed  by  FFAT  can  exceed  the  ESS  required  to  achieve  goals, 
thus  the  FFAT  is  self  defeating  for  HDBK  344. 

vi)  The  recommendation  is  to  use  SPC  techniques  and  the  results  of 
the  entire  ESS  to  demonstrate  that  defects  remaining  are 
'acceptable’.The  FFAT  would  be  used  to  represent  some  minimum 
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Uvel  of  ESS  that  would  need  to  be  retained,  even  if  results 
indicated  that  ESS  could  be  eliminated  in  its  entirety.  Although 
the  goal  of  ESS  is  to  eventually  make  ESS  unnecessary  (i.e.  defect 
causes  are  identified  and  eliminated  and  latent  defects  driven  to 
extinction)  a  certain  amount  of  stress  testing  is  re..>.ired  to 
demonstrate  the  process  is  in  control. 

3.5  Procedure  D  •  Cost  effectiveness  analysis. 

a)  Purpose.  To  optimize  the  cost  of  the  ESS  program 

b)  Comments. 

i)  The  procedure  analyses  the  cost  of  various  screening  options  (screen 
type  selection  and  placement)  to  identify  the  most  cost  effective 
plan.  The  concern  with  the  HDBK  procedure  is  philosophical  in 
that  the  HOBK  uses  a  cost  threshold  of  $1000/defect  to  represents 
field  repair  cost  and  coiild  have  the  mistaken  implication  that  if 
U‘«er’s  ESS  cost/defect  can  not  be  reduced  below  $1000/defect,  then 
the  defect  level  is  adequate  (4.10.3.7).  Also,  the  threshold  cost  can 
be  expected  to  vary  with  time  and  equipment,  eg.,  the  HDBK  also 
cites  10-15k$/defect  (ref.  HDBK  para.  4.4). 

ii)  The  HDBK  procedure  should  be  changed  to  emphasize  and  include 
the  following: 

-  reliability  requirements  must  be  achieved 

the  user  optimizes  the  cost  using  HDBK  344  to 

i)  prevent  defects 

ii)  screen  what*'  not  preventable 

SPC  techniques  are  employed  by  the  user  to 

i)  demonstrate  that  requirements  are  being  achieved 

ii)  strive  for  quantitative,  continuous  improvement 
beyond  compliance  ba^d  on  economic  trade  offs 
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The  ESS  plan  should  predict  the  'marginal’  cost  of  defect  elimination 
(i.e.  the  cost  of  screening  the  last  factory  defect)  so  that  if  the 
specified  reliability  can  not  be  economically  achieved,  the  customer 
and  producer  can  assess  the  economic  tradeoffs  of  relaxing  reliability 
requirements  for  cost  considerations. 

The  HDBK  procedure  performs  the  cost  analysis  on  a  program  life 
basis  (i.e.,  total  systems  to  be  produced).  The  recommendation  is  to 
change  this  to  a  per  sy  stem  basis. 

3.6  Procedure  E  -  Monitorine.  Evaluation,  and  Control. 

a)  Purpose.  To  monitor,  evaluate  and  control  ESS  tiad  manufacturing  to 
assure  that  the  requirements  for  remaining  defects  are  achieved 

b) 

i)  Monitoring  evaluation  and  control  using  SPC  techniques  are 
fundamentally  necessary  for  an  effective  ESS  Program. 

The  problems  that  need  to  be  addressed  are 

a)  what  parameters  should  be  monitored 

b)  how  can  these  parameters  be  measured  and 

c)  how  do  they  relate  to  customer  reliability  or  cost  requirements, 

ii)  The  purposes  of  monitoring  and  control  are 

a)  to  ensure  the  outgoing  product  meets  reliability  requirements 

b)  to  ensure  in-house  TQM  goals  are  being  met  and 

c)  identify  the  cause  and  correcti^’e  action  for  out  of  control  conditions. 


97 


tii)  The  hey  elements  of  management  monitoring  are  thus  a  set  of 
performance  indices  and  established  goals  (and  warning  limits)  for 
these  indices  and  a  means  of  comparing  actual  performance  with  these 
requircme  4*s.  The  critical  parameters  for  monitoring  and  control  are 
thus  Ortnutnlng*  Screening  strength,  Defect  Density,  Defects  Removed, 
and  Coat  Since  only  observahle  statistics  are  fallout  (ref.HDBK  para. 
4.11.4  i.c.  DR  (defects  removed))  procedures  must  be  provided  to 
determine  values  for  the  parameters  of  interest  from  the  factory 
fallout 

3.7  Procedure  El. 

a)  Purpose.  To  estimate  Din  and  SS  (procedure  A  and  B)  to  make 
inferences  regarding  remaining  defects. 

b)  Comments.  From  foregoing  discussions  and  the  HDBK  (ref.  4.11.4.3) 
the  actual  and  predicted  values  of  Din  and  SS  can  differ  significantly. 
Since  neither  Din  ncr  SS  can  be  evaluated  using  procedure  El,  the 
procedure  has  limited  and  questionable  use  and  procedure  E2/E3  is 
preferred. 

3.8  Procedure  E2/E3. 

a)  Purpose.  To  estimate  Din,  SS,  and  (Dremaining)  using  a  curve  fitting 
methodology. 

b)  Comments. 

i)  In  principle  this  technique  is  similar  to  that  used  in  the  study 
contract.  To  be  practical  though,  the  user  needs  software  tools  to 
assist  in  the  curve  fitting  analysis.  The  tools  must  be  flexible  since 
several  options  and  criteria  are  possible  for  curve  fitting  and  there  is 
not  one  unique  or  l}9St*  solution.  For  the  study  contract,  LSL 
developed  PC  software  to  assist  in  the  curve  fitting  analysis. 

ii)  Because  there  is  no  single  solution  with  the  curvefitting 
methodology,  analyses  made  by  the  same  user  are  reasonably  self 
consistent;  however,  comparison  of  ansJyses  made  by  different 
users  requires  caution. 


iii)  The  mathematical  model  for  curve  fitting  must  include  a  patent, 
defect  term  and  constant  failure  rate  and  may  be  required  to 
model  time  dependent  detection,  thus,  the  Chance  Defective 
Exponential  (CDE)  model  needs  enhancement 

3.9  Procedure  E  SPC  Charts. 

a)  Purpose.  To  demonstrate  that  the  process  is  under  control  (i.e. 
reliability  is  being  achieved)  or  conversely  identify  when  the  process  is 
not  in  control  and  additional  ESS  and/or  special  action  is  required. 

b)  Comments. 

i)  The  HDBK  SPC  charts  are  based  on  the  nonconformances  for 
individual  systems.  A  recommended  change  is  to  track  the 
normalized  statistic  eg.  defects  per  unit  (DPM  or  DPU) 
calculated  from  the  average  defects  per  system  in  a  batch  of  N 
systems  since  these  parameters  are  more  directly  related  to 
ESS  parameters  and  can  track  performance  with  selectable 
sensitivity. 

ii)  Conventional  SPC  charts  are  often  a  test  of  homogeneity.  Por 
ESS  purposes,  defect  densities  are  expected  to  improve 
(through  corrective  actions  to  eliminate  defect  causes);  thus,  it 
is  appropriate  to  track  the  moving  process  capability  and  use 
statistical  significance  tests  to  determine  if  the  process  is  out 
of  control.  The  recommended  SPC  chart  was  described  in  Part 
B. 

iii)  It  is  not  adequate  to  merely  demonstrate  control.  The  process 
must  be  capable  of  achieving  the  required  field  reliability. 
The  reliability  requirements  must  thus  be  translated  into  a 
corresponding  factory  requirement  and  included  on  the  SPC 
chart 
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A  Failure  Free  Acceptance  Test  (FFAT)  is  proposed  in  the 
HDBK  as  a  means  of  'ensuring’  that  the  remaining  defects 
are  below  the  required  level  without  actually  estimating  their 
magnitude.  The  FFAT  is  undesirable  from  an  ESS  and 
producer’s  aspect;  however,  if  FFAT  is  not  used,  an  acceptable 
viable  alternative  to  FFAT  is  required. 

The  SPC  control  techniques  can  be  directly  used  for  this 
purpose.  The  use  of  the  curve  fitting  software/methodology 
not  only  determines  the  Dpat.  Dlat.  SS,  etc.  but  also 
inherently  estimates  the  remaining  defects.  The  remaining 
defects  can  be  charted  on  similar  SPC  control  charts  to  those 
discussed  above  and  thus  used  to  indicate  compliance  with 
requirements.  The  requirement  for  Dremaining  would  be 
established  using  the  method  described  previously. 

The  SPC  approach  is  recommended  over  the  FFAT  since  it  is 
consistent  with  TQM,  and  allows  the  user  to  reduce  incoming 
defect  density  (Din)  an  alternative  to  screening  and  to 
tailor  the  factory  ESS  program  based  upon  measured  OiN  and 
the  field  reliability  rec^uirements. 


4.  PROPOSED  CHANGES  TO  THE  HDBK 

This  section  provides  a  summary  of  the  purposes  changes  to  the  handbook. 

4.1  The  proposed  changes  to  the  handbook  are  illustrated  in  the  revised  model 
shown  in  Fig  4.1.  The  proposed  changes  do  not  affect  the  basic  concepts  and 
methodology  described  in  the  HDBK  and  the  incoming  defects  per  system  are 
calculated  in  a  manner  similar  to  the  current  handbook.  The  complexity  of  the 
system  is  described  by  the  number  of  items  in  various  type-reliability  grade 
categories.  The  defects  per  sjrstem  are  then  calculated  by  multiplying  each  of 
these  complexity  values  by  the  corresponding  defect  density  for  each  category. 
The  recommended  changes  however  modify  and  affect  the  estimate  of 
incoming  defects  as  follows: 

i)  Workmanship  complexity  and  thus  defects  are  determined  based 
upon  the  MIL-STD-2000  assembly  and  solder  complexity  numbers. 
This  change  was  made  to  improve  the  accuracy  of  the  estimated 
workmanship  defects. 

ii)  The  defect  population  (ie  parts  and  workmanship)  is  proportioned 
into  separate  populations  that  are  sensitive  to  RV  and  TC  stresses. 
ESS  calculations  are  subsequently  performed  on  these  separate 
populations.  This  change  was  made  to  improve  modelling  accuracy 
and  to  ensure  a  proper  balance  of  RV  and  TC  screens. 

iii)  The  defects  are  determined  relative  to  the  R&M  2000  stress  levels. 
These  stress  levels  are  defined  to  be  the  reference  or  baseline  stress 
levels.  Defect  densities  for  other  factory  ESS  stress  levels  are 
determined  by  multiplying  the  reference  values  by  an  appropriate 
Stress  Adjustment  Factor  (SAF).  The  values  of  field  defects  under 

.  different  operating  environments  calculated  using  the  defect 
densities  for  that  environment,  eg.  AIF,  etc. 
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4.2  The  calculations  of  defects  removed  and  defects  remaining  are  also  similar 
to  the  existing  handbook  in  that  the  defects  removed  are  calculated  by 
multiplying  the  system  (or  assembly  )  defect  density  by  the  applicable 
screening  strength.  The  recommended  changes  affect  the  procedures  as 
follows: 

i)  the  defects  removed  by  screening  are  calculated  relative  to  the 
baseline  stress.  The  actual  defects  removed  are  then  calculated  by 
multiplying  the  removals  by  an  appropriate  stress  adjustment 
factor. 

ii)  the  terminology  was  changed  from  test  strength  screening 
strength  x  detection  efficiency  to  screening  strength  = 
precipitation  efficiency  x  detection  efficiency.  This  change  was 
made  to  make  the  terminology  more  consistent  and  descriptive. 

iii)  precipitation  efficiency  is  determined  using  the  same  equations  as 
those  used  to  produce  to  present  HDBK  tables.  The  precipitation 
efficiency  for  RV  however  was  modified  to  include  an  axis 
sensitivity  factor.  This  change  was  made  to  improve  modelling 
accuracy  based  on  the  axis  sensitivity  observed  in  the  study. 

iv)  the  stress  parameters  eg  Grms,  Temperature  transition  rate  etc  are 
defined  relative  to  the  unit  under  test  and  not  the  environmental 
chambers  etc.  The  requirement  for  thermal  and  vibrational  surveys 
to  determine  appropriate  values  was  also  added.  (Consistent  with 
this  change,  the  stress  level  in  the  precipitation  efficiency  equation 
may  need  to  be  rescaled.) 

v)  the  requirement  to  calculate  the  damage  factors  due  to  the  ESS  was 
added  to  ensure  that  the  ESS  stress  levels  and  duration  are  not 
destructive  or  consume  a  significant  portion  of  the  useful  fatigue 
life. 

4.3  Further  changes  and  refinements  concerned  the  data  analysis  and  SPC 
procedures  and  the  requirement  for  FFAT. 


i)  The  procedures  were  modified  to  encourage  the  ma;umum  use  of 
observed  data.  Initial  estimates  of  defect  density  and  screening 
strength  are  made  using  the  HDBK/industry  data  base;  however, 
these  estimates  are  subsequently  refined  by  the  user  based  on  the 
actual  data.  The  methodology  provided  to  enable  the  user  to 
measure  the  ESS  parameters  (eg  defect  density,  screening  strength 
etc)  is  based  on  a  curve  fitting  solution  to  the  general  ESS 
mathematical  expression  developed  in  Appendix  A.  These  changes 
were  made  to  eliminate  the  need  for  highly  accurate  data  in  the 
HDBK. 

ii)  For  analysis  and  modelling  purposes  defects  are  segregated  into 
errors  and  defects  with  defects  being  further  subdivided  into  latent 
and  patent  defects.  Since  it  is  precipitated  latent  defects  that 
determine  the  reliability  in  the  field  it  is  important  to  distinguish 
between  errors  and  defects.  Although  the  user  must  minimize  and 
control  errors,  the  improvements  in  these  areas  do  not  necessarily 
reduce  latent  defects  nor  improve  reliability. 

iii)  The  SPC  control  charts  used  for  monitoring  purposes  were  modified 
to  show  requirements  that  are  based  on  and  directly  related  to  the 
customer's  reliability  requirements.  In  addition,  the  process  mean 
is  determined  using  regression  analysis  since  the  mean  is  expected 
to  change  as  a  result  of  corrective  actions  and  continuous 
improvement.  A  modiHed  form  of  PARETO  chart  is  also 
recommended  to  help  identify  problems  requiring  analysis.  The 
modification  to  the  PARETO  is  to  not  only  compare  on  the  basis  of 
frequency  of  occurrence  but  to  relate  the  frequency  to  that  expected 
based  on  the  unit’s  complexity  and  the  ESS  predictions. 

iv)  The  mathematical  expression  described  in  Appendix  A  is  used  to 
relate  remaining  defects  (at  ESS  stress  levels)  to  field  reliability. 
This  relationship  requires  prior  knowledge  of  the  average  time 
constant  in  the  field.  Alternatively,  if  the  actual  stress  levels  are 
known,  the  precipitation  efficiency  equations  can  be  directly 
applied.  With  either  method,  the  original  estimates  are  to  be 
refined  based  on  actual  data. 
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v)  The  requirement  for  a  failure  free  acceptance  test  (FFAT)  was 
eliminated  and  replaced  with  an  SPC  program  to  measure  and 
control  remaining  defects.  The  FFAT  requirement  was  considered 
to  be  potentially  damaging  and  uneconomical  and  tended  to  be 
contrary  to  ESS  and  the  HDBK  philosophy  of  defect  elimination 
and  control.  A  minimum  verification  test  is  used  however  so  that 
ESS  can  not  be  entirely  eliminated  and  tests  remain  in  place  to 
collect  SPC  data. 

4.4  As  part  of  the  contract,  LSL  developed  PC  software  to  facilitate  the 
implementation  of  the  HDBK  procedures.  The  software  provided  in  the  tool  kit 
included  planning  software  (344PLAN),  data  analysis  software  (344CURVE) 
and  SPC  software  (344SPC  and  344PARTO).  A  tool  to  assist  in  the  planning 
phase  ie  344CHART  was  also  provided  to  prepare  a  multi  level  flow  chart.  This 
software  and  its  operation  was  explained  in  a  separate  wftware  users  manual. 


5.  ILLUSTRATIVE  EXAMPLE 

This  sectidn  illustrates  the  application  of  the  HDBK  methodology  proposed  in 
this  report  The  example  makes  use  of  the  PC  software  developed  for  this  study 
contract  and  contains  example  screens  produced  by  the  various  programs  in 
the  tool  kit 

5,2  In  planning  an  ESS  program  it  is  necessary  to  determine  the  factory 
screening  strength  required  to  satisfy  customer  reliability  requirements. 

This  is  determined  for  a  given  design  (ie.  complexity)  and  level  of  naaturity  and 
is  initially  planned  using  HDBK  344  data  or  relevant  experience  data,  and 
subsequently  refined  using  actual  data  for  the  specific  system.  The  basic  steps 
for  planning  and  implementing  an  ESS  program  are  summarized  in  Table  5.1, 


TABLE  5.1.  ESS  Planning  Steps. 


TASK 

DATA 

1.  Determine  initial  defects  per 
system 

-  Complexity  of  system 

•  Defect  Density  for  Parts  and 
Workmanship 

•  Limiting  Constant  Failure  Rate 
(CFR) 

2.  Determine  customer  reliability 

-  MTBF  and/or  FFOP 

-  W arranty  or  field  defect  costs 

-  Operating  life  and  field  conditions 

3.  Define  assembly  and  test 
restrictions 

-  Test  Flow  restrictions  for 
integration  calibration, 
contractual  ATP 

4.  Determine  required  factory 
screening  strength 

-  Precipitation  Efficiency  Equations 
Detection  Efficiency 

5.  Optimize  ESS  selection  and 
placement  based  on  cost 

-  Cost  model;  test  and  rework 
fatigue  damage  assessment 

6.  Create  TOals  and  Performance 
Indices  for  Monitoring  &  TQM 

7.  Refine  Estimates  ofDIN.SS  and 
undate  ESS  Program  based  on 
Observed  Results 

-  FRACAS  data 

5.2.1  To  plan  an  ESS  Program,  the  user  must  know 

i)  the  required  field  MTBF  (or  failure  rate),  as  specified  by 
the  customer 

ii)  the  limiting  failure  rate  of  the  system  based  on  relevant 
data  from  similar  equipments. 

(iii)  the  system  complexity  defined  in  terms  of  the  part  types 
and  reliability  grades  and  the  MIL-STD-2000  soldering 
and  assembly  complexity  values. 

5.2.2  For  this  example  assume 

i)  the  required  field  MTBF  =  3400  hrs. 

ii)  limiting  MTBF  =  10,000  hrs.  (CFR  =  10-4) 

the  allowable  MTBF  due  to  latent  defects  is  thus, 

■  1/(1/3400  - 1/10000)  =  5151-hrs. 

The  example  uses  5500  to  provide  a  safety  factor. 

5.3  Before  performing  any  the  calculations,  the  user  must  specify  the 
modelling  parameters  eg  fraction  of  RV  and  TC  sensitive  defects  and  the 
factory  ESS  stress  adjustment  factors  for  RV  and  TC,  as  illustrated  in  Figure 
5.1. 

5.4  The  first  step  is  to  determine  the  estimated  incoming  defects  per  sjrstem. 
To  perform  this  calculation  the  user  defines  the  system  parts  and 
Workmanship  complexity  for  each  assembly  (i.e.  complexity  vector)  and  thus 
the  complete  system,  (i.e.  complexity  matrix) 

Figure  5.2  illustrates  the  loading  screen  for  an  example  assembly  and  Figure 
5.3  shows  the  complexity  of  a  complete  system. 

5.4.1  The  user  then  multiplies  the  complexity  vectors  by  the  defect  density 
vector  to  determine  the  expected  parts  and  workmanship  defects  relative  to  the 
application  environment,  in  this  example,  AIF.  Figure  5.3  shows  the  results  of 
an  example  calculation. 
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Note  that  the  displayed  data  includes  the  total  system  defects  (i.e.,  1.124)  as 
well  as  the  sum  of  the  individual  assembly  defects.  Because  some  assemblies 
have  multiple  usage  on  the  system,  the  system  defects  can  be  greater  than  the 
sum  of  the  individual  assembly  defects.  Both  values  are  useful  for  monitoring 
and  control  and  were  thus  calculated.  For  the  example  shown,  they  are  equal. 
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k'lGURE  5.3.  F.Tgmple  System  CoiP^lexitv  and  Defects  for  AIF 

Environment. 


The  display  also  indicates  the  total  test  cost  (at  this  stage  without  ESS)  and  the 
proportion  of  parts  and  workmanship  defects. 

Parts  =  .806/1.124  =  .72 

Workmanship  =  .318/1.124  =  .28 

5.5  The  next  Step,  is  to  determine  the  screening  strength  required  to  meet  the 
customer  required  reliability. 

5.5.1  The  customers  MTBF  requirement  is  translated  into  a  requirement  for 
remaining  defects.  This  calculation  uses  expression  2.2  with  user  speciHed 
values  for  t  and  K,  in  this  example  t=:  1000  Hr  and  K  =  1/1000.  The  ratio  of 
defects  remaining  to  initial  defects  determines  the  screening  strength 
required.  If  the  detection  efliciency  is  known  then  the  precipitation  ef^ciency 
can  be  calculated  and  the  equations  for  precipitation  efficiency  solved  to 
determine  the  required  RV  andTC  as  illustrated  in  Figure  5.4. 

5.5.2  To  detennine  the  factory  defect  levels  corresponding  the  field  defect 
density,  the  user  must  apply  a  stress  adjustment  factor  (SAF).  The  SAF 
relating  the  field  and  baseline  stress  is  calculated  by  repeating  the  defect 
density  calculations  of  5.4  using  the  baseline  stress  (factory)  environment,  as 
shown  in  Figure  5.5,  and  taking  the  rati  ^  of  the  defects  per  system  at  the  field 
and  baseline  stress.  For  this  example  the  SAF  is  1.124/2.726  =  0.41. 

5.6  To  calculate  the  defects  removed  by  ESS,  the  screening  strength  must  be 
calculated  as  the  product  of  the  precipitation  efficiency  and  the  detection 
efficiency.  Precipitation  efficiency  is  determined  using  the  equations  given  in 
the  HDBK  for  the  type  of  environment  and  stress  levels  used  for  ESS.. 
Detection  efficiency  (DE)  considers  the  factors  summarized  in  Figure  5.6. 
(Note  that  DE  is  low  if  testing  is  not  performed  concurrently  with  stress.)  The 
defects  removed  are  determined  by  multiplying  the  screening  strength  by  the 
incoming  defect  density.  This  calculation  should  be  performed  separately  for 
TC  and  RV  defects. 
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FIGURE  5.6.  F.rample  Test  Detection  Efficiency  Calculation. 
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5.7  The  ESS  plan  as  designed  above  would  meet  reliability  requirements  but 
would  not  be  optimized  for  cost  Assemblies  with  high  screening  cost  ie  cost  of 
removing  a  defect,  should  be  considered  as  candidates  for  module  level  or 
special  assembly  level  ESS.  Adding  ESS  at  lower  levels  and  repeating  the 
planning  process  in  an  iterative  manner  determines  the  cost  optimized  ESS 

.  plan  with  consideration  for  the  integration,  calibration  and  functional  testing 
restrictions  required  to  manufacture  the  product.  An  example  of  an  optimized 
ESS  program  is  illustrated  in  Figure  5.7.  Note  that  the  system  complexity  has 
been  converted  into  defect  densities  (DPU)  for  the  various  parts  and 
complexity  factors.  These  values  as  well  as  those  for  total  defects  and  defects 
removed  would  be  used  as  requirements  for  SPC. 

5.8  Equation  2.2  can  then  be  used  to  verify  that  field  reliability  (MTBF)  will 
be  achieved  and  to  estimate  the  FFOP.  The  results  of  such  a  calculation  are 
illustrated  in  Figure  5.8. 

5.9  To  ensure  that  the  ESS  program  is  not  too  stressful  and  does  not  consume 
too  much  of  the  useful  (fatigue)  life,  the  user  computes  damage  factors  for  RV 
and  TC.  The  life  capabilities  can  be  determined  from  design  requirements, 
qualification  test,  or  the  anticipated  end  application.  These  calculations  are 
illustrated  in  Figure  5.9. 

5.10  Actual  production  data  would  be  subsequently  analyzed  to  refine  the 
screening  strength  and  defect  density  estimates.  This  is  accomplished  by 
curvefitting  the  defect  removal  distribution  to  expression  2.2,  and  is 
illustrated  in  Figure  5.10. 

5.11  SPC  would  be  used  to  monitor  and  control  the  critical  ESS  parameters  ie 
defects  remaining,  defect  density,  screening  strength,  defects  removed  and  cost 
etc.  Illustrative  SPC  and  PARETO  charts  are  shown  in  Figures  5.11  and  5.12. 
Note  that  these  charts  included  the  requirements  as  well  as  actual  and 
statistically  expected  variation. 


117 


5 


1 


2 


1  Sum  of  DeCAssy  at  baaaliae  stress,  excluding  effect  of  multi-usage  assy’s. 

2  Total  defects  removed  at  actual  ESS  stress,  including  effect  of  multi-usage 
assy’s. 

3  Defects  remaining  at  baseline  stress,  including  effect  of  multi-usage  assy’s. 

4  Cost  ofESS,  including  normal  testing. 

5  Defects  per  system  due  to  indicated  part  k  grade. 


FIGURES.?.  FTAmTilP!  Defecte  R^oved  and  Remaining  and  ESS  Goals  for 

nntimiTpd  Plan. 


118 


FXBLO  NTBF 

Factory  Stroas  Fiald  Straaa 

DBF(ro»ainliig)  latant  0.619  Daf/Sya  0.254  Daf/Sya 

DBF(raaatning)  patant  0.071  Daf/Sya  -  Daf/Sya 

STRESS  FACTOR  (FXBLD/FACTORT)  0.41 

Conataot  Fallora  Rata  (fiald  atraaa)  l.OOB-04  Daf/Br 

Fiald  tiaa  eoaatant  1000  Bra 


OPBRATIBG  BRS 

Dpat 

Dlat 

Dcfr 

Dtot 

NTBF 

%  FFOP 

10 

0.071 

0.003 

0.001 

0.074 

135 

92.85% 

100 

0.071 

0.024 

0.010 

0.105 

954 

90.05% 

200 

0.071 

0.046 

0.020 

0.137 

1464 

87.23% 

300 

0.071 

0.066 

0.030 

0.166 

1803 

84.674 

500 

0.071 

0.100 

0.050 

0.221 

2267 

80.21% 

1000 

0.071 

0.160 

0.100 

0.331 

3020 

71.81% 

3000 

0.071 

0.241 

0.300 

0.612 

4904 

54.24% 

5000 

0.071 

0.252 

0.500 

0.823 

6077 

43.92% 

%  Dafact  Fraa  Syatan,  axelading  CFR  affact  ia  72% 


FIGURE  5.8.  Example  Calculation  of  Expected  Field  MTBF  and  FFOP. 
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figure  5.9.  Example  Fatigue  Life  Calculation. 
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FIGURE  5.11.  Example  SPC  Chart. 


6.  CONCLUSIONS 


The  methodology  of  DOD  HDBK  344  was  successfully  applied  to  three 
diH'ereut  types  of  equipment  in  an  actual  factory  environment.  The 
equipments  were  designed  by  different  companies  and  were  in  manufacture  at 
LSL  on  a  build  to  print  basis  and  thus  represented  a  larger  cross  section  of 
milita^’y  electronic  hardware  and  designs.  Although  the  basic  methodology  of 
the  HDBK  was  verified  in  this  study,  the  detailed  procedures  needed 
modification  to  be  practical  and  to  improve  modelling  accuracy.  The  rationale 
and  details  of  these  changes  were  discussed  in  this  report  and  are  reflected  in 
recommended  changes  to  the  HDBK  and  PC  software  developed  under  this 
study  contract  to  facilitate  ESS  implementation. 

In  making  the  recommended  changes,  the  level  of  accuracy  required  of  the 
HDBK  and  the  complexity  of  the  model  were  kept  in  perspective  and 
minimized  by  incorporating  changes  that  allowed  the  maximum  use  of  user 
and  industry  data.  Recognizing  the  rapid  advances  in  technology  and 
associated  changes  in  defect  densities,  the  accuracy  of  HDBK  tables  she.  .Id  be 
suffleient  for  initial  prediction  and  planning  purposes  only,  with  higher  levels 
of  accuracy  being  achieved  using  actual  data  to  refine  the  estimates.  Many  of 
the  modelling  changes  had  the  intent  of  preventing  the  inadvertent 
misapplication  of  the  HDBK  and  included  the  partitioning  of  defects  into  RV 
and  TC  sensitive  populations  and  the  inclusion  of  a  stress  adjustment  factor  in 
the  general  model.  Nonetheless,  many  of  the  generic  observations  cited  in  the 
HDBK  as  well  as  the  precipitation  efTiciency  equations  demonstrated 
reasonable  correlation  with  the  observed  findings. 

Historically,  failure  mechanisms  that  have  occurred  ear’/  in  the  field  have 
been  of  a  similar  nature  and  cause  as  those  found  through  factory  ESS.  The 
PCDBK  s  use  of  ESS  to  stimulate  and  either  eliminate  the  cause  of  the  defect 
mechanisms  and/or  control  the  magnitude  (defect  density)  are  inherently 
fundamental  to  providing  the  field  with  reliable  electronic  hardware.  The 
quantitative  approach  provided  by  the  HDBK  is  a  viable  methodology  for 
providing  assiirance  that  field  reliability  will  be  achieved  and  provides  TQM 
with  the  necessary  capability  to  establish  factory  requir?  J'®nts  that  can  be 
measured  and  directly  related  to  customer’s  measures  of  satisfaction. 
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APPENDIX  A 

A.I.  INTRODUCTION 

A.1.1  This  Appendix  describes  the  aspects  of  Environmental  Stress  Screening 
[ESS]  theory  and  modeling  that  are  fundamental  for  an  effective  ESS  program. 

A.1.2  The  following  aspects  of  ESS  modelling  are  addressed: 


i) 

Basic  ESS  model 

Paragraph  A.2 

ii) 

Test  Detection  Efficiency 

Paragraph  A.3 

iii) 

ESS  model 

Paragraph  A.4 

iv) 

Effect  of  increased  stress 

Paragraph  A.5 

V) 

Concept  of  Stress  Hardening 

Paragraph  A.6 

vi) 

Conclusions 

Paragraph  A.7 

A.2.  BASIC  ESS  MODEL 

AJ2.1  For  ESS  to  be  a  viable  methodology,  it  is  necessary  that  the  screening 
strength  of  a  screen  be  independent  of  the  number  of  defects.  Also  the 
screening  strength  must  be  independent  of  when  the  screen  is  performed  and 
the  screening  strength  of  n  screens  of  duration  S  must  be  the  same  as  one 
screen  of  duration  n8. 

A.2.2  Mathematically,  these  requirements  can  be  satisfied  if  the  defects  are 
exponentially  distributed  in  time. 

Dx  =  Di(t)(l-e*K8)  where  Dx  is  defects  precipitated 

Di  (t)  is  latent  defect  population  at 
time  t 

K  is  stress  factor 
8  is  screen  duration 
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A.2.2.1  Since  Di(t)  is  the  latent  defects  remaining  at  time  t 

then  Di(t)  =  Dinitial  6'^^  where  Dinitial  is  the  number  of  latent  defects  at 

time  =  0. 


~  ^initial  ( 1  *  e-KS)  for  all  t  and  the  screening  strength 
is  given  by 


Thus,  for  the  exponential  distribution,  screening  strength  is  independent  of  t 
and  is  given  by  1-e-k® 

A.2,3  The  other  fundamental  requirement  for  ESS  is  that  the  screening 
strength  of  n  screens  of  duration  S  must  be  the  same  as  one  screen  of  duration  n 
8. 

Assuming  the  exponential  model,  the  screening  strength  is  1  -  e-KS. 

The  screening  strength  of  n  successive  screens  is  thus 

=  i.e-n6t 


This  is  the  same  as  the  screening  strength  of  duration  nS;  hence,  the 
exponential  model  satisfies  the  fundamental  requirements  for  ESS. 

A.2.4  A  direct  consequence  of  this  model  is  that  the  latent  defect  population 
has  a  constant  hazard  rate  K 


Hazard  rate  = 


Failure  rate 


Population  remaining  D  —  D(1  —  e“*^b  De~*'‘^ 


=  K 


A.2.4. 1  Assuming  that  the  defect  rate  of  the  normal  (good)  population  is 
constant  i.e. 


MTBF  = ,  — ;  Ag  =  constant  =  NC 
Ag 


where  N  is  the  complexity  factor  determined  from  the  number  of  parts  and 
workn>anship  operations  and  C  is  the  defect  density  for  each  of  the  complexity 
factors.Tlie  defect  rate  of  the  combined  ’good’  and  *bad’  populations  is  given  by; 

DEFECTS  REMOVED  =  D,  (1  -  e~ +  NC  t 

d  ■ 

hence  the  defect  rate  =  —  (Defect  Removed) 
dt 

=  DiKe-Kt  +  NC 

i.e.,  the  defect  rate  decays  exponentially  with  time  and  is  asymptotic  to  NC. 

It  should  be  noted  that  this  result  is  the  same  as  the  CDE  model  given  in  the 
handbook;  however,  whereas  the  CDE  model  assumes  a  Xg  and  Xb,  this 
derivation  is  based  only  upon  fundamental  and  essential  requirements  for 
ESS. 

A.2.4.2  If  ESS  is  to  be  mathematically  described  by  screening  strength,  it  is 
necessary  to  verify  that  defect  precipitation  is  exponential.  The  curve  Htting 
analysis  performed  in  the  study  contract  did  verify  that  the  exponential 
distribution  is  a  reasonable  representation  of  the  actual  conditions. 

A.3.  TEST  DETECTION  EFFICIENCY 

A.3.1  Assuming  that  defects  have  been  precipitated,  it  is  necessary  to  detect 
(i.e.,  observe)  the  defect. 

The  test  detection  efiiciency  DE  is  essentially  a  measure  of  the  ability  to  detect 
a  patent  defect  and  hence  assumes  a  degree  of  controllability  of  the  affected 
circuitry  and  observability  of  the  defect. 

A.3.1.1  In  practice,  the  controllability  and  observability  are  a  function  of 
i)  system  design  and  application 
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ii)  test  equipment 


iii)  test  environment 

A.3.2  An  important  consideration  is  that  defects  may  appear  to  be 
intermittent  and  also,  what  constitutes  a  defect  for  one  system/application 
may  not  appear  to  be  a  defect  in  another,  eg.,  FMECA.  As  heuristic  examples 
consider  the  following: 

i)  catastrophic  failure  (open)  of  a  capacitor  may  not  adversely  afTect 
performance  if  the  capacitor  is  used  for  redundant  power  supply 
filtering;  however,  the  same  capacitor  used  in  a  timing 
application  could  cause  system  failure. 

\ 

ii)  a  broken  lead  or  cracked  solder  joint  may  increase  in  resistance  or 
open  circuit  (momentarily)  with  temperature  and/or  vibration.  If 
the  connection  is  part  of  a  high  impedance  path  or  a  heavily 
filtered,  low  frequency  signal  path,  performance  may  not  be 
degraded;  however,  the  same  defect  involved  in  high  speed  data 
transmission  or  clock  signals  could  cause  system  failure. 

iii)  soft  (i.e.,  momentary)  errors  such  as  those  that  occur  in  memories 
or  data  transfer,  may  not  be  perceived  as  a  problem  in  a  display 
application  eg.  cause  an  imperceptible  flicker  on  a  display,  but 
could  be  a  failure  in  a  navigation  system  where  parametric 
accuracy  is  important  or  if  the  signal  were  used  for  control 
purposes  eg.  system  shut  down,  etc. 

In  each  of  the  3  foregoing  examples,  it  should  be  noted  that  the  failure  could  be 
repeatable  and  easily  detected  or  be  of  an  intermittent  nature. 

A.3.3  These  examples  serve  to  illustrate  that: 

i)  fault  detection  may  require  extended  testing  to  detect  and  isolate 
the  fault. 
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ii)  fault  detection  may  require  environmental  stresses  eg.  vibration, 
.  temperature,  temperature  change. 

iii)  the  detectability  ofthe  fault  is  a  function  of  the 

a)  type  and  application  of  the  equipment 

b)  the  ability  of  the  test  equipment  to  exercise  the  necessary 
function  i.e.,  controllability  and  to  detect  the  failure  i.e. 
observability. 

iv)  the  recognition  of  a  defect  is  influenced  by  its  criticality. 

A.3.4  An  additional  consequence  is  that  two  systems  with  the  identical 
complexities  and  exposed  to  identical  screens  could  appear  to  have  different 
latent  defect  populations  because  of  the  following: 

i)  equipment  type  and  purpose/function 

ii)  type  of  test  equipment 

iii)  nature  of  test 

and  iv)  criticality  of  defectCs) 

A.3.5  Mathematically,  the  time  dependency  aspect  of  detection  efficiency 
would  appear  to  lend  itself  to  a  Poisson  distribution. 

DE  =  DP  (1  -  e-K2t)  were  DP  (the  probability  of  detection)  and  K2  (the 
detection  stress  constant)  are  measures  of  the  inherent  test  detection 
capability. 

A.3.6  Considerations  not  directly  related  to  detection  efficiency  but  affecting 
its  perception  are: 

i)  the  ability  to  isolate  and  repair  defect,  and 


ii)  the  ability  to  repair  the  defect  without  introducing  additional 
defects. 

A.3.7  The  factors  to  be  considered  in  assessing  test  detection  eSiciency  are 
summarized  in  Table  A.1. 

TABLE  A.l.  Factors  Affecting  Test  Detection  Efficiency. 
Probability  of  Occurrence. 

Fault  Detection  (with  and  without  concurrent  stress). . 

Test  Duration  (with  and  without  concurrent  stress). 

Fault  Isolation. 

Repair/Rework  defect  creation  density. 

A.3.8  For  effective  fault  detection,  extended  test  times  under  varying  stress 
types,  eg.  RV,  TC,  etc.,  are  required  in  conjunction  with  test  equipment 
capable  of  exercising  all  functions  and  observing  any  defects.  Subsequent  to 
fault  detection,  the  defect  must  be  isolated  and  repaired  without  introducing 
additional  faults. 

A.4.  MORE  COMPLETE  MODEL 

A.4.1  The  problems  associated  with  defect  precipitation  and  detection  become 
interrelated  since  the  same  environmental  stresses  necessary  to  precipitate  a 
defect  are  also  involved  in  its  detection.  In  other  words,  during  defect 
detection,  defect  precipitation  is  also  occurring,  and  vice  versa. 

A.4.2  This  can  be  expressed  mathematically  as  the  convolution  of  the 
precipitation  and  detection  functions. 

A.4.2. 1  Assuming  precipitation  3is  a  function  of  stress  level  (K)  and  duration 
(t)  is  given  by  Dx  (t,K)  =  Di  (1  -  e-Kt)  and  detection  is  given  by 
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Dg(t  K)  =s  Dp(  1  -  e  -K2t)  then  the  observed  defect  rate  is  given  by 

Di(  1  -  e-Kit)  ♦  (Dp(l  -  e-K*t)  where  ‘denotes  convolution 

KAJIJI  To  solve  this  it  is  convenient  to  use  the  fact  that  convolution  in  the 
time  domain  is  the  inverse  Fourier  (or  Laplace)  transform  of  the  product  of  the 
Fourier  (or  Laplace)  transforms  of  Dx  and  DE 

Defect  rate  =  F-HF(Dx)F(DE)] 

A.4.3  At  this  point,  it  is  useful  to  observe  the  analogy  to  electronic  signals.  In 
essence,  defect  removal  has  two  aspects;  a  precipitation  term  that  is  analogous 
to  a  generation  term  and  a  detection  term  that  is  analogous  to  a  (low  pass) 
filter  term. 

A.4.4  The  following  conditions  are  appropriate  for  ESS  modelling  ^d  are 
solved  in  Table  A-2.  _  _  -  — 

i)  detection  of  an  existing  precipitated  defect  population  eg. 
testing  after  non-operating  screen, 

ii)  detection  of  defects  precipitating  with  an  exponential 
distribution  i.e.,  during  ESS. 

ii)  detection  of  detects  precipitating  with  a  constant  rate  i.e., 
due  to  limiting  MTBF  of  System  (design). 

A.4.5  The  more  complete  ESS  model  requires  the  summation  of  the  3  terms 
identified  above  and  is  as  follows: 
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Dp  =  EXISTING  PATENT  DEFECTS  AT  START  OF  ESS 

Di  =  LATENT  DEFECTS  AT  START  OF  TEST 

Dc  =  'CONSTANT  FAILURE  RATE’  DUE  TO  A  LIMITING 
MTBF. 

De=  test  detection  EFHCIENCY 
B  =  TEST  DETECTION  EFFICIENCY  •K’ FACTOR 
A  =  ESS  DEFECT  PRECIPITATION  'K’  FACTOR 
NOTE; 

The  subscripts  1,2  and  3  refer  to  patent, 
precipitating  latent,  and  constant  failure  rate  effects 
respectively. 


TABLE  A.2.  Mathematical  Expression  for  ESS. 


GENERATION  TERM 
LAPLACE 


DETECTION 


OFEXISTINO 

POPULATION 


or  PATENT  OCFECTS 


DOMAIN  TRANSFORM  LAPLACE 

TRANSFORM 

D  ^  ^ 

S  S  +  B 


TIME  DOMAIN 
RESPONSE 


Dd  - 


FRECZPrrATmo 


SXPOKtZfrtAt 

oommmoK 


D(1  -  «■•*') 


S(S  + A)  S  +  B 


A*  B.Dll-B 


(B  -  A)  CA  -  B) 


A  =  B,Z)l|l-e''*‘[Al+  1)1 


puoprrATmG 

ATCOrCSTANTRATE 

(i.«.  CoaiUBt  Xl 


5  +  B 


D  —St 
D 


A.4.5.1  The  effect  of  these  equations  was  shown  in  Part  B  Figures  2.9  and 

2.10. 

A.4.6  In  this  model,  the  effect  of  defects  in  the  replacement  population  is 
inherently  included  in  the  exponential  rate  model. 

A.5.  EFFECT  OF  INCREASED  STRESS 

A.5.1  Assume  that  latent  defect  removal  can  be  modeled  mathematically  as 
follows 


Dremoved  =  Dl(x)(l-e-tK(x))  where  Di(x)  total  latent  defects, 

K(x)  is  stress  factor,  t  is  stress 
duration,  and  z  is  stress  level. 

A.5.2  Increasing  stress  can  then  have  the  3  possible  effects  indicated  in 
Table  A.3. 


TABLE  A.3.  Effect  Of  Increasing  Stress  on  Defects  and  *K*. 


Possibility 

Total  Latent 
Defects 

K 

Comment 

A 

Increase 

Increase 

Most  General 

B 

Increase 

No  change 

Unlikely/ESS 

hnpractical 

C 

No  change 

Increase 

Simplest  to  apply 

A.5.3  Consider  first  possibility  A  since  this  is  the  most  general  case  from 
which  possibilities  B  and  C  can  be  derived.  With  this  possibility,  increasing 
the  stress  level  increases  both  the  total  number  of  latent  defects  and  the 
precipitation  factor  K. 


A.5.3  Defect  mechanisms  that  lend  tb^-mselves  to  this  model  (possibility  A) 
are  mechanical  type  failures  due  to  stress  or  fatigue  limits.  Due  to  the 
multitude  of  connection;,  joints  and  members  in  the  equipment,  each  with  a 


possibility  of  various  imperfections,  the  equipment  will  experience  increased 
failures  as  the  stress  level  increases  towards  the  stress/fatigue  limits  of  the 
imperfections. 

A  Ji.4  Possibility  B,  which  has  only  latent  defects  increasing  with  stress  level 
would  mean  that  ESS  is  ineffective  since  the  defect  precipitation  rate  can  not 
be  accelerated  by  applying  a  higher  stress  level.  This  hypothesis  can  be 
rejected  based  on  available  evidence  that  indicates  that  precipitation  can  be 
accelerated. 

A.5.S  Possibility  C  has  a  fixed  number  of  latent  defects  (for  stress  levels  kept 
within  design  capabilities);  however,  the  precipitation  rate  can  be  accelerated 
by  increased  stress. 

Many  defect  types  lend  themselves  to  this  model.  For  example  reactions  with 
acceleration  factors  that  have  an  Arrhenius  relationship  i.e.  acceleration 
factor  of 


apply  since  the  failures  would  occur,  regardless  of  stress  level,  and  are  only 
accelerated  by  increased  stress. 

In  general,  all  reaction  rates  that  increase  with  temperature  are  included  in 
this  group.  Failures  resulting  from  chemical  reactions,  corrosion, 
intermetallic  formation,  diffusion,  leaks,  or  contaimnation  ingress,etc.,  should 
thus  lend  themselves  to  this  ESS  model  (Possibility  C). 

A.5.6  Both  models  A  &  C  thus  have  technical  merit  and  need  to  be  used  in  ESS 
modelling,  with  A  representing  the  most  general  case. 

The  consequences  of  Model  A  are  that  increasing  stress  causes  increased 
defects  (beyond  what  is  necessary  or  would  occur  in  the  end  application)  which 
have  a  repair  cost  that  affects  the  economical  optimization  of  an  ESS  program. 


A.6.  THE  CONCEPT  OF  STRESS  HARDENING 

A.6.1  Stress  Hardness.  To  provide  insight  into  defect  precipitation  and  thus 
the  validity  of  various  ESS  models  and  assumptions,  the  concept  of  'STRESS 
HARDElilNG’  has  been  developed.  Stress  hardening  assumes  that  defects 
have  a  distribution  in  strength  (for  each  stress  type).  Qualitatively  this  can  be 
explained  as  follows: 

i)  Defects  with  a  lov/  strength  precipitate  quickly  under  low  applied 
stress,  eg.,  infant  mortality. 

ii)  Defects  with  greater  strength  require  higher  stress  or  longer  time 
to  precipitate. 

Stress  hardening  thus  involves  removing  defects  below  a  stress  level  such  that 
they  do  not  (readily)  precipitate  in  the  field.  Stress  hardness  is  thus  the  ability 
to  withstand  a  given  stress  level,  as  measured  by  the  defect  precipitation  rate. 

A.6.2  Stress  Hardness  Model.  The  Stress  Hardness  Model  assumes  that 
flaws  have  a  distribution  in  strength  and  that  under  an  applied  stress,  the 
flaws  weaken  and  migrate  to  lower  strength  levels  according  to  an  Arrhenius 
relationship. 

Et 


Defects  with  no  residual  strength  are  patent  defects. 

An  example  of  a  stress  hardness  analysis  is  provided  in  Figures  A.I. 

Figure  A.1  is  the  result  of  a  uniform  distribution  in  stress  and  wear  out 
populations.  Note  the  general  shape  of  the  familiar  'bath  tub’  curve  and  that 
the  initial  reduction  in  defect  rate  is  nearly  exponential.  Subsequent  to  the 
fiat,  constant  failure  rate  period,  the  failure  rate  increases  due  to  wear  out. 
The  first  wear  out  affects  only  part  of  ^e  population,  the  second  wear  out  is 
more  extensive. 
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A.6.6^  In  practice,  every  interconnect  and  joint  .etc.,  will  have  a  multitude  of 
possible  failure  mechanisms  with  varying  stress  hardness  capability.  The 
total  number  of  defects  possible  ^thin  a  system  thus  becomes  very  large. 

A.6.6.4  Depending  upon  the  defect  distribution,  the  model  predicts  that  the 
defect  rate  decreases  with  time  and  is  asympitotip  tc  a  limiting  defect  rate  that 
is  a  function  of  the  stress  level.  This  is  consistent  with  the  models  developed  in 
paragraphs  2  and  4  and  generally  accepted  reliability  theory. 

A.7.  CONCLUSIONS 

A.7.I  From  the  discussion  paper,  several  aspects  of  ESS  modeling  and 
analysis  have  been  addressed  that  a^ect  the  perception  of  screening 
effectiveness  and/or  ESS  prediction  accuracy. 

A.7.2  The  precipitation  and  detection  of  defects  can  be  modeled  as  in 
paragraph  A4. 

A.7.2. 1  Since  both  defect  precipitation  and  detection  aie  exponential,  the 
perception  of  screening  strength  may  be  a  measurement  of  detection  efliciency, 
depending  upon  which  K  term  dominates. 

A.7.2.2  Since  stress  and  time  are  required  for  defect  detection,  the  value  and 
economics  of  lower  level  ESS  need  to  be  assessed. 

A.7.3  The  estimation  of  test  detection  efficiency  is  as  important  as  the  need  to 
estimate  precipitation  efficiency.  Test  detection  efficiency,  however,  is 
affected  by  several  considerations,  as  discussed  in  paragraph  A3  and  is  not 
trivial. 

A.7.4  A  proper  ESS  model  recognizes  that  (  certain  types  of )  latent  defects 
increase  with  stress  level.  A  priori  knowledge  of  the  defect  vs.  stress 
distribution  is  thus  necessary. 
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A.7.5  The  perception  of  defects  is  affected  by  the  type  and  design  of  the 
equipment  and  how  it  is  used  and  tested.  Thus  equipment  with  similar 
complexities  may  appear  to  have  different  latent  defects  depending  on  the 
testing  and  application. 

A.7.6  The  concept  of  stress  hardening  tends  to  explain  observed  ESS  and 
reliability  effects  from  a  simple  model  and  could  be  a  useful  tool  or  concept  for 
ESS  modeling.  The  model  predicts  that  flaws  will  have  a  temporal 
distribution.  If  the  stress  is  high  with  respect  to  the  flaw  strength  then  a 
certain  flaw  size  will  cause  failures  that  have  a  normal  distribution  in  time 
becoming  log  normal  for  greater  flaw  strengths.  This  prediction  of  the  model  is 
consistent  with  the  literature  that  indicates  that  failures  often  have  a  log 
normal  distribution  in  time.  This  means  failures  are  not  random  but  are 
caused  by  a  definite  defect  mechanism  that  has  both  a  strength  distribution 
and  temporal  distribution  when  under  stress. 

A.8.  DEFECT  TYPES  AND  DEFINITION 

A.8.1  For  consistent  ESS  modelling  and  terminology,  it  is  necessary  to 
examine  the  definition  of  defects  based  upon  type  and  impact  on  ESS.  In 
general  defects  belong  to  2  major  groups,  latent  and  patent. 

A.8.1.1  LATENT  DEFECT:  defect  that  has  the  potential  of  eventually 

causing  a  failure,  i.e.,  flaw  with  a  residual 
strength. 

A.8.1.2  PATENT  DEFECT:  defect  that  has  the  potential  of  immediately 

causing  a  failure,  i.e.,  flaw  with  no  residual 
strength. 

NOTE: 

It  should  be  noted  that  not  all  latent  nor  patent  defects 
will  necessarily  cause  a  failure. 

A.8.2  Latent  Defects  for  ESS  purposes  are  of  2  types. 
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A.8.2.1  Stress  dependent  latent  defects.  Latent  defect  that  precipitates  as 
a  patent  defect  only  if  a  threshold  stress  is  exceeded.  The  existence  of  stress 
dependent  latent  defects  results  in  an  increasing  number  of  defects  with 
increasing  (ESS)  stress  levels.  Because  ESS  inherently  involves  different 
stress  levels  and  stress  level  tradeoffs,  it  is  fundamentally  necessary  to  be  able 
to  model  and  predict  this  type  of  defect  (i.e..  Possibility  A  in  A.5). 

A.8.2.2  Stress  Independent  Latent  Defects.  Latent  defect  that  precipitate 
as  a  patent  defect  regardless  of  stress  level.  Increasing  stress  has  the  effect  of 
accelerating  the  time  to  failure;  however,  the  total  quantity  of  defects  does  not 
increase  (i.e..  Possibility  C  in  A.5). 

A.8.3  Patent  defects  need  to  be  considered  based  upon  their  origin  and 
detectability.  Patent  defects  that  are  due;  to  assembly  and  workmanship 
'errors’  must  be  distinguished  from  patent  defects  due  to  precipitated  latent 
defects.  The  purpose  of  ESS  is  to  i)  identify  (and  thereby  possibly  eliminate 
through  corrective  action)  the  latent  defect  cause  and  ii)  reduce  the  number  6f 
latent  defects  through  screening  and  thereby  reduce  the  failure  rate. 

i)  Errors.  Errors  are  patent  defects  that  are  caused  by  workmanship  and 
are  readily  detected  and  removed  by  simple  testing  or  inspection.  ESS  is 
thus  not  required  to  detect  and  eliminate  errors  and  thus  • -  ors  have 
only  an  indirect  effect  on  reliability  due  to  rework. 

Errors  should  be  preventable  and  should  not  occur  whereas  patent 
defects  arising  from  latent  defects  are  only  preventable  to  the  limits  of 
the  state  of  the  art  in  technology  and  equipment. 

Although  errors  should  be  monitored  using  SPC  and  efforts  taken  to 
eliminate  them,  the  data  and  associated  SPC  should  be  separated  from 
patent  defects  resulting  from  a  latent  defect. 

ii)  Patent  Defects.  Although  errors  are  also  a  subset  of  patent  defects,  for 
terminological  simplicity,  patent  defects  can  be  considered  as  only  those 
resulting  from  latent  defects. 


A.8.4  Using  the  suggested  definition  of  patent  defects,  ESS  modelling  needs  to 
recognize  2  types  as  determined  by  their  detectability. 

i)  Stress  Detectable  Patent  Defects.  Defects  that  although  having  no 
residual  strength  cannot  be  detected  without  the  application  of  stress 
and  time.  Examples  include  fractured  wires  making  (intermittent) 
contact,  etc.  Detection  of  these  defects  requires  testing  concurrent  with 
stress  application,. 

ii)  Immediately  Detectable  Patent  Defects.  Defects  that  can  be  detected 
without  requiring  stress,  provided  the  proper  test  is  performed. 

A.8.5  Although  perhaps  appearing  complex  in  definition,  it  is  important  that 
the  above  types  of  defects  be  recognized  in  ESS.  From  a  practical  aspect, 
though,  it  is  recommended  that  the  defect  definition  be  simplified  and 
combined  as  shown  in  Table  A.4. 


TABLE  A.4.  Defect  Definition  Simplification. 


ESS  DEFECT 

SUGGESTED  HDBK 
DEFINITION 

COMMENTS 

LATENT,  STRESS 
DEPENDENT 

LATENT 

Combine  all  3  defect 
types  provided  HDBK 
models  stress  dependent 
flaws. 

From  a  practical  aspect, 
stress  detectable  patent 
is  equivalent  to  a  latent 
defect 

LATENT,  STRESS 
,  INDEPENDENT 

PATENT,  STRESS 
DETECTABLE 

PATENT,  ERROR 

ERROR 

PATENT, 

IMMEDIATELY 

DETECTABLE 

PATENT 
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A.8.6  Screenable  Defect. 

A.8.6.1  From  a  practical  aspect,  a  defect  (mechanism)  should  be  only 
considered  screenable  if  precipitating  the  defect  reduces  the  failure  rate.  From 
a  causwl  aspect,  this  would  mean  that  a  non  screenable  defect  would  be  one 
that  the  stress  required  to  remove  that  flaw  would  cause  the  strength  of  a 
lesser  flaw  to  be  degraded,  such  that  there  would  be  no  net  improvement. 

Because  of  this  effect,  continued  ESS  approaches  a  screening  limit,  beyond 
which  failure  rate  is  not  improved  even  though  defects  are  being  removed. 

This  effect  is  predicted  and  demonstrated  by  the  stress  hardening  model  of 
paragraph  A5.6. 

It  is  also  illustrates  that  for  optimum  reliability  the  design  and  manufacturing 
processes  must  be  proper  and  controlled,  and  that  there  is  a  limited  reliability 
that  can  be  achieved  through  screening  alone. 


A.8.7  Relevant  and  Non  Relevant  ."defects.  ESS  can  remove  defects  that 
improve  Held  reliability;  hence,  these  defects  are  relevant .  Because  of  stress 
dependent  latent  defects  and  the  screening  limit,  ESS  can  remove  defects  that 
do  not  improve  field  reliability,  i.e.,  would  not  have  occurred  under  field  stress 
conditions;  hence,  these  defects  are  non  relevant  It  is  necessary  that  ESS 
modeling  predict  and  separately  track  relevant  and  non  relevant  defects.  Non 
relevant  defects  resulting  from  stress  levels  that  are  too  high  (and  possibly 
damaging)  increase  the  production  cost  due  to  rework  and  influence  ESS 
optimization  and  stress  limitations.  From  a  practical  aspect,  the  ESS  estimate 
of  latent  defects  should  include  only  relevant  defects  ,  i.e.,  defects  above  the 
screening  limit. 
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DOD-HDBK-344  Software  Toolkit  and  User  Manual 

B.l  Description:  DOD-HDBK-344  contains  procedures  for  the  planning, 
monitoring  and  control  of  a  cost-effective  ESS  program.  These  procedures 
and  associated  data  bases  for  part,  assembly,  and  soldering  defect 
densities,  equations  to  calculate  screening  strength,  and  tools  for  pre¬ 
paring  multilevel  flow  charts,  curvefitting  actual  data,  and  preparing  SPC 
and  PARETO  diagrams  have  been  Incorporated  into  a  suite  of  PC  software 
programs.  This  software  runs  in  a  LOTUS  1-2-3  environment  and  operates 
interactively  with  the  user.  The  user  interface  is  through  user  friendly 
menus  and  screens  that  are  controlled  and  created  by  LOTUS  macros  that 
operate  transparently  to  the  user.  In  this  mode,  a  comprehensive  under¬ 
standing  of  LOTUS  1-2-3  is  not  required.  If  the  user  is  familiar  with 
LOTUS,  some  of  the  executions  can  be  perfonned  directly  from  LOTUS  if 
desired.  The  software  can  thus  be  used  by  personnel  with  a  minimum  of 
experience  on  PC  computers  and  LOTUS  1-2-3,  and  is  offered  in  a  spreadsheet 
environment  that  is  widely  used  throughout  industry.  The  software  toolkit 
for  implementing  all  of  the  procedures  of  DOD-HDBK-344  is  made  up  of  the 
following  programs; 

,  (a)  344MMASTR  - 

(b)  344PLAN 

(c)  344CHART 

(d)  244SPC 

(e)  344CURYE  _ 

(f)  344PART0 

Each  of  the  programs  described  below  is  a  shell  that  contains  the 
necessary  equations,  viewing  screens,  and  macros  for  both  specific 
execution  steps  and  operator  interface.  Data  required  by  the  program, 
{e.g.  factory  defect  data  etc.)  are  loaded  into  the  worksheet  shell  from 
flat  (ASCII)  files,  other  worksheets,  and/or  by  direct  user  interface.  The 
program  performs  the  necessary  data  manipulations  and  calculations  and 
creates  output  data  files  for  use  by  the  operator  or  other  344xxx  software 
programs.  These  files  can  be  text  files  or  other  worksheets  depending  on 
the  application.  File  transfers  are  commanded  by  the  user,  but  the  type  of 
file,  etc.  is  transparent  to  the  user  with  the  program  determining  the 
necessary  type  of  file  and  its  structure. 

344MASTR  is  the  control  program  that  guides  and  instructs  the  user  on  which 
program  to  use  for  specific  tasks.  The  working  program  listed  above  can  be 
called  up  and  loaded  directly  from  344MASTR.  A  brief  discussion  of  ESS  and 
introductory  help  screens  on  the  purpose  and  use  of  344xxx  programs  are 
also  provided. 

344PLAN  is  the  program  used  to  plan  and  optimize  the  ESS  program.  This 
program  implements  Procedures  A,  B  and  C  of  DOD-HDBK-344.  The  program 
contains  help  screens  on  the  basic  operation  of  the  program  and  contains 
utility  modules  for  estimating  field  reliability,  determining  damage 
indices,  and  estimating  detection  efficiencies.  The  program  calls  up 
selected  records  from  the  data  base  of  part,  assembly  and  soldering  defect 
densities  (PPM)  that  are  contained  in  the  data  base  file  DEFDENSI(ty).WKl. 
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The  program  Is  used  during  the  initial  planning  phase  and  on  a  continual 
basis  to  refine  the  ESS  program  and  to  ensure  that  it  remains  optimum.  The 
program  creates  output  data  files  containing  the  goals  and  requirements 
for  parts  and  manufacturing  defects  for  each  assembly  and  level  of  integra¬ 
tion-test. 

344CHART  is  used  to  produce  a  multilevel  flow  chart  showing  defects 
entering  and  removed  at  each  assembly  screening  level.  The  chart  produced 
by  this  program  provides  the  framework  for  screen  selection  and  placement 
optimization.  Defects  Introduced,  removed  and  remaining  are  shown  for 
each  assembly  at  all  integration  test  levels.  Random  vibration  (RV) 
sensitive  and  temperature  cycling  (TC)  sensitive  defects  are  displayed 
separately  on  the  same  diagram.  Screening  and  test  costs  are  also 
indicated  to  help  identify  high  cost  screens  that  could  possibly  be  more 
cost-effectively  performed  at  an  earlier  stage.  The  data  for  this  program 
is  created  by  344PLAN. 

344SPC  is  used  to  prepare  statistical  process  control  (SPC)  charts  and 
implements  DOD-HDBK-344  Procedure  E.  These  are  modified  SPC  charts  that 
compare  present  performance  capability  with  requirements  that  relate  to 
the  required  field  reliability.  The  charts  thus  indicate  requirements, 
show  the  present  level  of  performance  (capability),  and  the  statistically 
expected  variation  due  to  limited  lot  size.  The  process  capability  is 
automatically  determined  by  the  program  through  a  polynomial  regression 
analysis  of  the  actual  data.  The  data  for  this  program  is  loaded  from 
actual  factory  or  field  data  either  manually  or  from  a  separately  prepared 
data  file. 

344PART0  is  used  to  prepare  PARETO  diagrams  for  any  desire  integration  test 
level.  These  are  modified  PARETO  charts  in  that  actual  results  are  not 
only  compared  on  the  basis  of  relative  frequency  of  occurrence,  but  also 
with  consideration  for  the  expected  frequency  based  on  the  relative  com¬ 
plexity  and  the  established  goals  and  requirements.  The  requirements  data 
base  is  created  and  updated  on  a  continual  basis  using  344PLAN.  The  actual 
factory  data  are  loaded  manually  or  from  a  separately  prepared  data  file. 

344CURVE  is  used  to  analyse  actual  factory  or  field  data  using  a  curve¬ 
fitting  solution  that  determines  the  critical  ESS  parameters  (e.g.  Din, 
SS,  Dremaining,  Dpatent,  etc.).  Data  for  this  program  are  loaded  manually 
or  from  a  specially  formatted  data  file.  The  results  of  this  analysis  are 
monitored  and  trended  using  344SPC  and  indicate  whether  or  not  field  reli¬ 
ability  will  be  achieved  and  to  what  extent  factory  T(3M  goals  are  being 
realized. 

B . 2  Requests  for  Copi es :  Government  agencies  may  obtain  copies  of  DOO- 
HDBK-344,  Software  Toolkit  and  Users  Manual  by  completing  The  Statement  of 
Terms  and  Conditions  provided  in  Section  B.3  below  and  sending  it  to 
RL/ERSR  (344  Toolkit),  Griffiss  AFB  NY  13441-5700.  In  addition,  DoD 
Contractors  are  required  to  submit  DO  Form  2345,  Military  Critical  Tech¬ 
nical  Data  Agreement  which  can  be  obtained  from  the  Defense  Logistics 
Service  Center,  Federal  Center,  Battle  Creek,  Michigan  49017-3084. 
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B.3  Statement  of  Terms  and  Conditions 


STATEMENT  OF  TERMS  AND  CONDITIONS  RELEASE  OF  AIR  FORCE  OWNED  OR  DEVELOPED 


COMPUTER  SOFTWARE  PACKAGES 


1.  Release  of  the  following  US  Air  Force  software  package  (computer  pro¬ 
grams,  systems  descriptions,  and  documentation)  Is  requested: 


2.  The  requested  software  package  will  be  used  for  the  following  purpose: 


Such  use  Is  projected  to  accrue  benefit  to  the  Government  as  follows: 


3.  I/We  will  be  responsible  for  assuring  that  the  software  package 
received  will  not  be  used  for  any  purpose  other  than  shown  In  Paragraph  2 
above;  also.  It  will  not  be  released  to  anyone  without  prior  approval  of 
the  Air  Force.  Further,  the  release  of  the  requested  software  package  will 
not  result  In  competition  with  other  software  packages  offered  by 
coninerclal  firms. 

4.  I/We  guarantee  that  the  provided  software  package,  or  any  modified 
version  thereof,  will  not  be  published  for  profit  or  in  any  manner  offered 
for  sale  to  the  Government;  It  will  not  be  sold  or  given  to  any  other 
activity  or  firm,  without  the  prior  written  approval  of  the  Air  Force.  If 
this  software  Is  modified  or  enhanced  using  Government  funds,  the  Govern¬ 
ment  owns  the  results,  whether  the  software  Is  the  basis  of,  or  Incidental 
to  a  contract.  The  Government  may  not  pay  a  second  time  for  this  software 
or  the  enhanced  or  modified  version  thereof.  The  package  may  be  used  In 
contract  with  the  Government  but  no  charge  may  be  made  for  Its  use. 

5.  The  US  Air  Force  Is  neither  liable  nor  responsible  foi'  maintenance, 
updating  or  correcting  any  errors  In  the  software  provided. 

6.  I/We  understand  that  no  material  subject  to  national  defense  security 
classification  or  proprietary  rights  was  Intended  to  be  released  to  us. 
I/We  will  report  promptly  the  discovery  of  any  material  with  such  restric¬ 
tions  to  the  Air  Force  approving  authority.  1/We  will  follow  all  Instruc¬ 
tions  concerning  the  use  or  return  of  such  material  In  accordance  with 
regulations  applying  to  classified  material,  adn  will  make  no  further 
study,  use,  or  copy  such  material  subject  to  security  or  proprietary  rights 
marking. 
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7.  I/We  understand  that  the  software  package  rece<ved  Is  Intended  for 
domestic  use  only.  It  vrlll  not  be  made  available  to  foreign  Governments 
nor  us6d  In  any  contract  with  a  foreign  Government 


Signature  of  Requestor  ST^ature  of  Air  Force 

Approving  Authority 


name  or  Requestor  - Rame/lltle  of  Air  Force 
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roroword 

1.  This  Handbook  providas  tsohniques  for  planninq  and  evaluating  Environmental 
Stress  Screening  (ESS)  programs.  The  guidance  contained  herein  departs  from  other 
approaches  to  ESS  in  that  quantitative  methods  are  used  to  plan  and  control  both 
the  cost  and  effectiveness  of  ESS  programs.  Handbook  procedures  and  methodology 
were  developed  under  RADC  contractual  and  in-house  studies.  Contractual  efforts 
were  performed  by  the  Hughes  Aircraft  Company  of  Fullerton,  California,  under  the 
direction  of  Nr.  A.  I.  Saari  and  Litton  Systems  Canada  Limited  of  Toronto,  Ontario 
under  the  direction  of  Mr.  R.  A.  Pepperall.  The  Handbook  includes,  the  guidance 
contained  in  RtN  2000  ESS  Policy  Letter  dated  2S  Jun  86. 

2.  Environmental  Stress  Screening  (ESS)  programs,  which  are  applied  during  the 

development  and  production  phases,  can  yield  significant  improvements  in  field 
reliability  and  reductions  in  field  maintenance  costa.  Application  during 

development  can  .reap  significant  savings  in  test  time  and  costs  as  a  result  of 
eliminating  or  reducing  the  number'  of  latent  defects  prior  to  qualification  tests. 
The  benefits  for  the  manufacturer  include:  a  high  degree  of  visibility  as  to  the 
sources, of  reliability  problems  in  the  product  or  process,  better  control  of  rework 
costa,  and  the  opportunity  to  determine  corrective  actions  which  eliminate  the 
sources  of  reliability  problems  from  the  product  or  process. 

3.  There  are  various  approaches  associated  with  the  application  of  stress  screens. 
Regardless  of  the  approach  used,  the  fundamental  objective  of  ESS  remains  the  same; 
i.e.,  to  remove  latent  defects  from  the  product  prior  to  field  delivery.  The 
quantitative  methods,  contained  in  this  Handbook,  extend  this  objective  by  focusing 
on  the  defects  which  remain  in.  the  product  at  delivery  and  their  impact  on  field 
reliability.  The  goal  of  ESS  programs  thus  becomes  to  reduce  the  latent  defect 
population,  at  delivery,  to  a  level  which  is  consistent  with  the  reliability 
requirements  for  the  product.  Reduction  of  the  latent  defect  population  in  a 
production  lot  of  electronic  equipment,  is  accomplished  by: 

a.  Use  of  ESS  to  precipitate  flaws  in  the  assembled  hardware  to  a  detectable 
level  coupled  with  the  use  of  thorough  tests  to  facilitate  their 
detection  and  removal. 

b.  Hse  of  ESS  results  to  isolate  defect-failure  causes  followed  by 
determining  appropriate  corrective  actions.  Effective , corrective  actions 
eliminate  the  source  (cause)  of  the  defect  from  the  process  or  product, 
thereby  improving  manufacturing  process  capability. 
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4.  G*n«r«l  guidelines  and  supporting  rationale  in  Section  4  and  detailed 
guidelines  in  Section  5  provide  the  user  with  the  procedures  needed  to  plan, 
monitor  end  control  the  screening  process  so  that  quantitative  goals  can  be 
achieved  cost  effectively.  The  six  detailed  procedures  of  Section  5  are  entitled: 

Procedure  A  •  Optimising  Screen  Selection  and  Placement 
Procedure  8  -  Estimating  Defect  Density 
Procedure  C  -  Estimating  Screening  Strength 

Procedure  D  -  Refining  Estimates  of  Defect  Density  and  Screening  Strength 
Procedure  E  -  Monitoring  and  Control 

Procedure  F  -  Product  Reliability  Verification  Test  (PRVT). 

5.  It  should  be  noted  that  is  is  not  possible  to  eliminate  all  defects  from  the 
hardware  through  stress  screening.  The  vast  majority  of  parts  in  the  hardware  will 
never  fail  throughout  the  life  of  the  product.  However;  some  fraction  of  the  parts 
contain  gross  latent  defects  and  tend  to  fail  early  and  thus  dominate  the 
reliability  of  fielded  products  during  early  life.  The  objective  is  to  remove  as 
many  of  the  gross  defects  from  the  hardware  as  is  technically  and  economically 
feasible  so  as  to  achieve  the  designed-in  or  required  reliability.  The  Handbook 
implements  these  objectives  through  use  of  controls  on  the  latent  defects  present 
in  the  hardware  at  assembly «  the  costs  to  precipitate  and  remove  them,  and  the 
assurance  needed  that  latent  defects  remaining  in  the  hardware  at  delivery  will 
allow  reliability  objectives  to  bo  achieved. 

6.  The  procedures  provided  in  the  Handbook  are  an  important  aspect  of  a 
manufacturer's  TQM  program  and  philosophy.  The  procedures . quantify  some  elements 
of  customers  satisfaction  that  are  measured  by  cost  and  reliability  and  reflect 
these  as  factory  goals  and  requirements  that  are  thus  meaningfully  and  directly 
related  to  the  customers  measures  of  satisfaction.  These  factory  requirements  apply 
to  all  levels  from  the  procurement  of  parts  and  materials  from  vendors  through  all 
factory  processes  and  tests  and  affect  both  management  and  design  philosophies.  The 
procedures  also  provide  management  and  working  level  groups  with  quantitative 
feedback  on  their  performance  compared  with  requirements  and  goals  for  continuous 
improvement.  If  problem  areas  or  deficiencies  are  identified  the  procedures  help 
analyse  options  for  defect  control  or  prevention. 
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1.  IMTRODUCTIOM 

1.1  Puroosa.  This  Handbook  prd"ides  uniform  procedures,  methods  and  techniques 
for  planning,  monitoring  and  controlling  the  cost  effectiveness  of  ESS  programs  for 
electronic  equipment.  It  is  intended  to  support  the  requirements  of  MIL-STD-78S, 
Task  301, "Environmental  Stress  Screening"  and  to  implement  Air  Force  RSM  2000  ESS 
recosnendations  and  guidelines. 

1.2  Application.  The  Handbook  is  intended  for  use  by  procuring  activities  and 
contractors  during  development  and.  production.  It  is  not  intended  that  the 
Handbook  procedures  and  techniques  be  used  in  a  cookbook  fashion.  Knowledge  of  the 
equipment  and  the  manufacturing  process  is  essential  for  a  properly  planned  and 
tailored  ESS  program.  The  data  base  needed  for  a  systematic  approach  to  ESS 
application  is  not  fully  developed.  Use  of  the  Handbook  by  Government  procuring 
agencies  and  equipment  manufacturers  will  foster  the  development  of  an  improved  and 
broader  data  base. 

1.3  General.  A  properly  applied  ESS  program  can  significantly  impact  the  quality 
and  reliability  of  electronic  products  delivered  to  the  Government.  ESS  is 
interrelated  with  the  requirements  set  forth  in  MIL-Q-98S8,  and  NIL'STD-78S. 
Quality  Control  is  a  manufacturing  function  and  Reliability  Engineering  is  a  design 
function.  Although  the  Quality  and  Reliability  disciplines  are  related,  in 
practice,  they  are  conducted  as  separate  programs  without  common  objectives.  The 
Handbook  uses  the  ESS  program  as  a  means  for  integrating  Quality  Control  and 
Reliability  Engineering  tasks  so  as  to  assure  achievement  of  reliability  objectives 
during  manufacture.  Supporting  software  is  available,  f^om  RAOC  that  fully 
automates  the  detailed  manual  procedures  contained  herein. 

1.3.1  What  is  ESS?.  ESS  is  a  process  or  series  of  processes  ,  in  which 
environmental  stimuli,  such  as  rapid  thermal  cycling  and  random  vibration,  are 
applied  to  electronic  items  in  order  to  precipitate  latent  defects  to  early 
failure.  An  equally  important  and  Inseparable  aspect  of  the  screening  process  is 
the  testing  which  is  done  as  part  of  the  screen,  to  as  to  detect  and  properly 
identify  the  defects  which  have  been  precipitated  to  failure.  The  precipitation 
and  testing  process  it  basically  a  search  for  defects.  Manufacturing  techniques 
for  modern  electronic  hardware  consist  of  hundreds  of  individual  operations  and 
processes  through  which  defects  can  be  introduced  into  the  product.  Many  of  the 
defects  can  be  detected  without  the  need  for  stress  screens  by  use  of  visual 
inspections,  functional  tests  and  other  conventional  quality  assurance  procedures. 
Such  defects  are  termed  errors  and  are  a  subset  of  patent  defects.  A  small 
percentage  of  latent  defects  remain  undetected  by  obvious  means  and,  if  not  removed 
in  the  factory,  will  eventually  manifest  as  early  life  failures  during  product  use. 
The  inability  to  find  latent  defects  by  obvious  means  is  a  consequence  of  the 
increased  eomplesity  of  modern  electronic  products  and  the  processes  which  are  used 
in  their  manufacture.  ESS  is  the  vehicle  by  which,  latent  defects  are  accelerated 
to  early  failure  in  the  factory.  ESS  can  thus  be  viewed  as  an  extension  of  the 
quality  control  inspection  and  testing  process. 

1.3.2  Organization  of  the  Handbook.  The  Introduction  (Section  1)  outlines  the 
purpose  of  the  Handbook  and  provides  general  introductory  remarks  pertaining  to  the 
quantitative  approach  to  ESS.  Section  2  lists  applicable  references  and  Section  3 
defines  terms  and  acronyms  used.  Section  4  contains  general  guidelines  and 
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provides  tho  catlonele  and  background  Cor  Che  detailed  guidelines.  Section  S 
contains  Che  detailed  guidelines  which  are  organized  according  to  the  sequence  of 
events  to  be  undertaken  by  the  contractor  in  planning,  monitoring  and  controlling  a 
screening  program.  Tho  detailed  procedures  are  entitled: 

Procedure  A  *•  Optimising  Screen  Selection  and  Placement 
.  Procedure  B  -  Bstimating  Defect  Density 
.  Procedure  C  -  Bitimating  Screening  Strength 

Procedure  0  -  Refining  fstimates  of  Defect  Density  and  Screening 
Strength 

Procedure  R  -  Monitoring  and  Control 

Procedure  P  -  Product  Reliability  Verification  Test 

Appendis  A  contains  the  mathematical  relations  and  model  descriptions  used  in  the 
Handbook.  A  review  of  Appendix  A  will  help  the  interested  reader  in  gaining  a 
quick  understanding  of  the  rationale  and  methodology  of  the  Handbook.  Appendix  B 
provides  the  mathematical  foundation  foe  the  Product  Reliability  Verification  Test. 

Figure  1.1  shows  the  sequence  of  application  of  the  various  tasks  contained  in  the 
Handbook  and  cross-references  them  to  the  applicable  procedures  of  the  Handbook. 

The  product  development  phase  is  used  to  experiment  with  stress  screens  to  refine 
tho  estimate  of  RSS  parameters  (Dm,  SS)  and  to  define  and  plan  a  coat  effective 
screening  program  for  the  production  phase.  The  incoming  latent  defect  density  is 
estimated  (Procedure  B)  and  screens  are  selectively  placed  at  various  assembly 
levels  to  develop  a  plan  for  achieving  quantitative  ISS  goals  cost-effectively 
(Procedure  A).  The  ESS  plan  for  the  development  phase  should  be  submitted  as  part 
of  the  Reliability  Program  Plan  (paragraph  4.4.1). 

An  ESS  plan  for  the  production  phase  is  submitted  based  upon  the  experimentation 
and  analyses  of  eostreffectiveness  (Para  4.4.1).  After  the  screening  program  is 
implemented  during  production,  the  fallout  from  the  screens  are  used  to  evaluate 
the  screening  process  and  to  establish  whether  ESS  program  objectives  are  being 
achieved  (Procedures  D  and  E).  Figure  1.2  shows  the  detailed  mathematical  model 
upon  which  the  ESS  program  is  based.  The  details  will  be  explained  as  the  reader 
continues. 

A  Product  Reliability  Verification  test  is  performed  and  the  results  used  in 
conjunction  with  data  from  the  entire  factory  ESS  program  to  provide  assurance  that 
quantitative  objectives  have  been  achieved  prior  to  delivery  to  the  customer 
(Procedure  F).  The  Quantitative  goals  for  the  screening  program  should  be 
established  in  accordance  with  the  methods  outlined  in  Procedure  A. 
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th«  MthcMtieal  aadal  can  be  tepreaented  by: 
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Figure  1.2  Mathanatical  Model  of  an  ESS  Program 
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1.3.3  Dgyelooment  and  Production  Phase  Reliability  Assurance.  ESS  is  not  a 
substitute  for  a  sound  reliability  program  conducted  during  the  design  and 
development  phases.  The  inherent  reliability  of  the  product  is  driven  primarily  by 
the  design.  However,  without  a  viable  reliability  assurance  program  during 
production,  the  reliability  which  is  designed  into  the  product  can  be  seriously 
degraded.  An  equipment  will  eventually  pass  a  MIL-STD-781  reliability 
demonstration  test,  either  during,  development  or  on  a  sample  basis  during 
production.  A  single  equipment  passing  the  MIL-STO-781  test  does  not  imply  that 
all  other  equipments  in  the  production  lot  have  the  same  reliability.  A  relatively 
few  latent  defects,  contained  in  various  equipments  in  the  lot,  can  significantly 
reduce  the  field  reliability,  especially  for  equipments  with  high  reliability 
requirements.  A  production  reliability  assurance  program  which  complements  the 
design/development  reliability  program,  is  therefore  essential  to  achieving  relia¬ 
bility  objectives.  A  properly  planned,  monitored  and  controlled  stress  screening 
program,  structured  as  part  of  a  production  reliability  assurance  program,  is  the 
vehicle  through  which  product  reliability  in  manufacture  can  be  maintained.  The 
identification  and  prevention  of  defect  .  causes  through  ESS  and  analysis  reduces 
defect  densities  for  production.  This  information  also  provides  feedback  to  a 
lessons-learned  data  base  to  avoid  similar  deficiencies  on  subsequent  designs  or 
Changes.  The  procedures  are  oriented  toward  achieving  reliability  objectives 
through  use  of  quantitative  methods  for  stress  screening  and  production  reliability 
assurance. 

1.3.4  ESS  Anolication  and  the  Quantitative  Approach.  Historically  there  have  been 
two  basic  approaches  to  the  application  of  stress  screens.  In  one  approach,  the 
Government  explicitly  specifies  the  screens  and  screening  parameters  to  be  used  at 
various  assembly  levels.  Failure-free  periods  are  sometimes  attached  to  the 
screens,  as  acceptance  requirements,  in  order  to  provide  assurance  that  the  product 
la  reasonably  free  of  defects.  Another  approach  is  to  have  the  contractor  propose 
a  screening  program  which  is  tailored  to  the  product  and  is  subject  to  the  approval 
of  the  procuring  activity.  Although  the  latter  approach  is  preferred,  neither 
approach  is  adequate  since  explicit  objectives  and  the  relations  between  the 
screening  program  and  quantitative  reliability  requirements  are  not  always  defined. 
Costs  are  also  uncontrolled  because  some  of  the  screens  might  be  more  efficiently 
performed,  at  lower  assembly  levels,  where  rework  costs  are  lower.  In  addition, 
screening  levels  may  far  exceed  the  design  limits  of  the  product  and  result  in 
damage  to  the  equipment. 

There  are  several  unknowns  associated  with  the  application  of  stress  screens.  How 
effective  are  the  screens?  What  is  considered  acceptable  or  unacceptable  fallout 
from  a  screen?  How  does  the  quantity  of  defects  remaining  in  the  equipment  after 
delivery  to  the  customer  impact  field  reliability?  The  aforementioned  ESS 
approaches  do  not  fully  address  these  questions.  For  example,  if  the  screen 
fallout  is  "low”,  it  is  not  known  whether  the  equipment  is  "good”  (i.e.,  defect- 
free)  or  whether  the  screen  is  not  effective.  On  the  other  hand,  if  the  fallout  is 
*high”,  it  is  not  known  whether  the  incoming  defect  levels  are  inordinately  high  or 
whether  the  screen  might  be  causing  non-defectives  to  fail. 

Screens  and  tests  are  not  perfect.  At  each  stage  of  manufacture  where  screens  and 
tests  night  be  applied,  from  device  level  to  the  final  system  level,  escapes  to  the 
next  assembly  stage  occur,  and  new  opportunities  for  introducing  defects  are 
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created.  The  nunber  of  latent  defects  which  remain  in  the  product  at  delivery  and 
their  impact  on  field  reliability,  however,  is  the  primary  concern. 

1.3. 4.1  The  Quantitative  Approach.  The  use  of  a  quantitative  approach  to  stress 
screening  requires  that  the  initial  part  latent  defect  levels,  the  defect  level 
introduced  during  manufacture  of  the  product,  the  effectiveness  of  the  screens,  and 
reasonably  acceptable  values  for  the  number  of  latent  defects  which  remain  and 
escape  into  the  field  be  addressed.  Figures  1.3  and  1.4  illustrate  the 
quantitative  aspects  of  stress  screening. 


Figure  1.3  The  Quantitative  Problem 


When  a  quantitative  approach  to  stress  screening  is  used,  the  key  variables  of 
interest  ace  the  average  number  of  defects  per  product  which  enter  the  screen  (Din 
comprised  of  latent  defects  (Ol)and  patent  defects  (Dp  and  E)},  the  screen  strength 
(SS)  which  is  the  product  of  Precipitation  Efficiency  (PS)  and  Detection  Efficiency 
(DE)  and  the  average  number  of  defects  per  product  which  escape  the  screen/test 
(Dremaimiiig) •  Figure  1.4  shows  the  relationships  between  these  stress  screening 
variables. 
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Figure  1.4  Stress  Screening  and  Variable  Relationships 
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The  number  oC  defects  remaining  in  the  production  lot  at  delivery  is  a  function  of 
three  key  factors: 

a.  The  quantity  of  design,  part  and  manufacturing  (workmanship  and 
process)  defects  which  initially  reside  in  the  hardware  prior  tc 
assembly  level  screening. 

b.  the  capability  of  the  environmental  stress  to  precipitate  flaws 
in  assemblies  to  a  detectable  level. 

c.  The  thoroughpess  of  the  testing  which  is  done,  either  during  oi 
after  the  screen,  to  assure  detection  of  the  defects  precipitated 
to  failure  by  the  screens  and  the  ability  to  fault  isolate  anc 
remove  the  defect  without  introducing  new  flaws. 

None  of  the  three  factors  which  impact  the  reliability  of  delivered  products  i: 
known  with  certainty.  Without  a  basic  knowledge  of  their  quantitative  value, 
however,  effective  screening  programs  cannot  be  properly  planned  and  controlled' 
The  procedures  in  the  handbook  are  directed  to  obtaining  both  preliminary  planninc 
and  measured  estimates  of  the  three  factors  in  order  to  plan,  monitor  and  control 
thO'  screening  process.  Experience  data  gathered  from  previous  screening  programs 
screening  experiments  conducted  during  the  development  phase  and  use  of  tht 
handbook  procedures  provides  the  methodology  and  inforsiation  needed  to  plan  am 
conduct  effective  screening  programs. 

Once  a  screening  program  is  implemented  during  production,  the  results  must  bi 
monitored  and  appropriate  changes  made  in  the  screening  regimen  to  assure  tha 
goals  on  remaining  defects  are  achieved.  The  basic  mechanism  for  assuring  contro 
is  to  compare  the  screening  results  with  established  goals  so  as  to  determine  th 
need  for  corrective  actions.  For  example,  corrective  actions  might  be  accomplishe' 
by  increasing  precipitation  or  detection  efficiencies  so  that  more  defects  can  b 
precipitated  and  detected,  or  by  reducing  incoming  defect  quantities  throug 
improved  process  controls.  Changes  which  reduce  or'  eliminate  screening  at  som 
levels  of  asrecbly  can  also  be  taken  to  reduce  costs,  when  it  is  found  that  th 
screens  are  ineffective  or  unnecessary. 

1.3.5  Benefits  of  a  Quantitative  Approach.  A  quantitative  approach  to  stres 
screening  enables  the  establishment  of  explicit  quantitative  objectives  an 
provides  a  basis  for  planning,  monitoring  and  controlling  the  screening  process  t 
meet  those  objectives.  A  quantitative  approach  also  facilitates  Government  an 
contractor  communication  on  the  status  of  the  screening  process  and  on  the  progres 
being  made  toward  achieving  objectives.  Coupled  with  a  good  Failure  Reportin 
hnalysis  aod  Corrective  Action  System  (FRACAS),  the  quantitative  approach  als 
provides  a  more  focused  emphasis  on  the  sources  of  latent  reliability  problems  i 
the  product  or  process  as  well  as  better  control  of  costs. 

1.3.6  Process  Capability  and  Defect  Density.  The  use  of  a  quantitative  approac 
to  stress  screening  requires  addressing  the  capability  of  the  manufacturing  proces 
to  produce  products  which  are  reasonably  free  of  defects.  Defects  are  introduce 
into  a  lot  of  manufactured  products  through  repeated  assembly,  handling  and  testir 
operations.  The  average  number  of  defects  per  product  (defect  density)  varies  as 
function  of  the  degree  of  control  which  is  exercised  over  tha  manufacturing  proces 
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and  thtf  process  capability.  The  ESS  program  addresses  the  questions;  What  is  the 
process  capability?,  What  must  the  process  capability  be  in  order  to  meet 
quantitative  reliability  objectives?  What  improvements  and  changes  are  required  to 
achieve  the  reliability  objedtives  at  optimum  cost? 


1-9 


DOO>HDBK-344  (USAF) 


2.  RZFERENCED  DOCUMENTS 

Th«  docuncnts  cited  in  this  section  see  for  guidance  and  information. 


2-1  Government  Documents. 

SPECIFICATIONS 

MIL-<1-98S8 

STANDARDS 

MIL-STO-72I 

MIL-STD-781 

Mir.-STD-785 

NIL-STO-883 

NIL-STO-2000 


Quality  Program  Requirements 


Definition  of  Terms  for  Reliability  and 
Maintainability 

Reliability  Design  Qualification  and 
Production  Acceptance  Tests:  Exponential 

Distribution 

Reliability  Program  For  Systems  and  Equipment 
Development  and  Production 

Test  Methods  and  Procedures  for 
Niecoelectronics 

Standard  Requirements  for  Soldered  Electrical 
and  Electronic  Assemblies 


HANDBOOKS 

HlL-HOBR-217  Reliability  Prediction  of  Electronic  Equipment 

PUBLICATIONS 


Air  Force 

AFMAL-TR-80-3086 
Aug  80 

RADC-TR-82-87 
May  82 
(AD-A118261) 

RAOC -TR-86-138 


Environmntal  Burn-In  Effectiveness 


Stress  Screening  of  Electronic 
Hardware 


RADC  Guide  to  Environmental  Stress 
Screening 


RAOC-TR-86-149 

Navy 


Environmental  Stress  Screening 


NAVMAT  P-9492 


Navy  Manufacturing  Screening  Program 
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(Copies  at  specifications,  standards,  handbooks,  drawings,  and  publications 
required  by  contractors  in  connection  with  specific  acquisition  functions  should  be 
obtained  from  the  contracting  activity  or  as  directed  by  the  contracting  officer.) 

2.2  Konoovernment  Documents. 

Institute  of  Environmental  Sciences  (lES) 

Environmental  Stress  Screening  Guidelines,  1981 

Environmental  Stress  Screening  Guidelines  for  Assemblies,  Sep  84 

(Application  for  copies  should  bo  addressed  to  the  Institute  of  Environmental 
Sciences,  940  East  Northwest  Highway,  Mt  Prospect  It  60056-3444) 

Electronic  Industries  Association  (EIA) 

Interim  Standard  No.  18  Lot  Acceptance  Procedure  for  Verifying 
Compliance  with  the  Specified  Quality  Level  (SQL)  in  PPM 

(Application  for  copies  should  be  addressed  to  the  Electronic  Industries 
Association,  2001  Eye  Street,  NM,  Washington  DC  20006  5009) 

2.2.1  Other  Nonr  vernment  Documents. 

Fertig,  E.W..,  Murthy,  V.E.,  "Models  for  Reliability  Growth  During 
Burn-In",  Proceedings  of  the  1978  Annual  RAM  Symposium,  pp  504- 
509. 

Bateson.  J.T.,  "Board  Test  Strategies  -  Production  Testing  in  the 
Factory  of  the  Future",  Test  and  Measurement  World,  pp.  118-129, 
Dec  84. 

Eube,  F.,  Hirschbergar ,  G.,'  "An  Investigation  to  Determine 

Effective  Equipment  Acceptance  Test  Methods",  Grumman  Aerospace 
Corporation,  Report  No.,  ADR  14-04-73,  Apr  73 

Brownlee,  K.A.  (1960),  Statistical  Theory  and  Methodology  in 
Science  and  Engineering,  New  York,  John  Wiley  and  Sons 

Crandall,  Random  Vibration,  John  Wiley  and  Son 

Engelmaier,  Effects  of  Power  Cycling  in  LCC,  Bell  Laboratories 
N .  J .  ■  , 

(Nongovernment  documents  are  generally  available  for  reference  from  libraries.  They 
are  also  distributed  among  nongovernment  standards  bodies  and  using  Federal 
agencies.) 
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3.  DEFINITIONS  AND  ACRONYMS 

3.1  Definitions.  Definitions  applicable  to  this  Handbook  are: 

Assenbly/Module  A  number  of  parts  joined  together  to  perform  a 

specific  function  and  capable  of  disassembly,  for 
eaaaqile  a  printed  circuit  assembly.  An  assembly  of 
parts  designed  to  function  in  conjunction  with 
similar  or  different  modules  when  assembled  into  a 
unit.  (eg.  power  supply  nwdule,  core  memory  module.) 

Baseline  Stress  Factory  ESS  stress  levels  consistent  with  RAM  2000 

guidelines  i.e.,  6  Grms,  2*C/min.  Measured  at  Unit 
Under  Test 

Chamber  Cabinet  in  which  hardware  is  placed  in  order  to  apply 

stress  to  it. 

Defect  Density  Average  number  of  latent  defects  per  item.  Symbols 

used:  Dm,  Dqut*  ^REMAINING  And  Dq  for  incoming, 

outgoing,  remaining  and  observed  defect  density, 
respectively. 

Detectable  Failure  A  failure  that  can  be  detected  with  100%  detection 

efficiency. 

Detection  Efficiency  A  measure  of  the  capability  of  detecting  a  patent 

defect.  Symbol  is  DE. 

Error  Class  of  patent  defect  resulting  from  assembly  and/or 

test  correlation  errors.  Errors  do  not  require 
environmental  stress  for  precipitation  or  detection. 

The  incoming  defect  density  which  is  not  detected  by 
a  screen  and  tent  and  which  is  passed  on  to  the  nest 
level. 

A  contiguous  period  of  tine  during  which  an  item  is 
to  operate  without  the  occurrence  of  a  failure  while 
under  environmental  sti.ess. 

Failure.  Rate  The  total  number  of  failuies  within  an  item 

population,  divided  by  the  total  number  of  life  units 
espended  by  that  population  during  c  particular 
measurement  interval  under  stated  conditions.  Symbol 
is  \.  A  reliability  measure  related  to  KTBF. 

Fallout  Failures  observed  during,  or  immediately  after,  and 

attributed  to  stress  screens.  Symbol  is  F. 
Sometimes  used  to  mean  defects  removed,  symbol 

dremoveo* 


Escapes 

Failure-Free  Period 
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Fault  Covcraqa  In.  a  givan  place  of  equipment,  the  ratio  of  faults 

which  are  detectable  to  faults  present. 

Latent  Defect  An  inherent  or  induced  weakness,  not  detectable  by 

ordinary  naans,  which  will  either  be  precipitated  to 
early  failure  under  environmental  stress  screening 
conditions  or  eventually  fail  in  the  intended  use 
enikironnont. 

Part  Any  identifiable  item  w^hin  the  product  which  can  be 

reamved  or  repaired  (e.g:,  discrete  semiconductor, 
resistor,  XC,  solder  joint,  connector). 

Part  Fraction  Defective  The  number  of  defective  parts  contained  in  a  part 

population  divided  by  the  total  number  of  parts  in 
the  population  expressed  in  PPM.  See  also  defect 
density. 

Patent  Defect  An  inherent  or  induced  weakness  which  can  be  detected 

by  inspection,  functional  test,  or  other  defined 
means.  Symbol  is  Dp^t*  In  this  procedure,  Dpat 
refers  to  precipitated  latent  defect.  See  also 
error.  . 

Precipitation  (of  Defects)  The  process  of  transforming  a  latent  defect  into  a 

patent  defect  through  the  application  of  stress 
screens. 

Precipitation  Efficiency  A  measure  of  the  capability  of  a  screen  to 

precipitate  latent  defects  to  failure.  Symbol  is  PE. 

Production  Lot  A  group  of  items  eanufactured  under  essentially  the 

same  conditions  and  processes. 

A  test  to  provide  confidence  that  field  reliability 
will  be  achieved. 

A  latent  defect  that  when  removed,  results  in  a  lower 
equipment  failure  rate. 

Screen  Parameters  Parameters  in  screening  strength  equations  which 

relate  to  screening  strength,  (  e.g.,  vibration  G- 
lovdls,  temperature  rate  of  change  and  time 
duration). 

Screening  Regimen  A  combination  of  stress  screens  applied  to  ah 

equipment.  Identified  in  the  order  of  application 
(i.e.,  assembly,  unit  and  system  screens). 

Screening  Strength  The  probability  that  a  specific  screen  .  will 

precipitate  a  latent  defect  to  failure  and  then 
detect  the  resultant  patent  defect,  given  that  a 


Product  Reliability 
Verification  Test 
Screenable  Latent  Defect 
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latent  defect  susceptible  to  the  screen  is  present. 
It  is  the  product  of  precipitation  efficiency  and 
detection  efficiency.  Symbol  is  SS. 


Selection  and  Placement 


Stress  Adjustamnt  Paetor 


Stress  Screening 


System/Equipment 


Thermal  Survey 


Vibration  Survey 


Yield 


The  process  of  systematically  selecting  the  mest 
effective  stress  screens  and  placing  them  at  the 
appropriate  levels  of  assembly. 

The  ratio  of  the  incoming  defect  density  at  the 
anticipated  field  stress  level  to  the  incoming  defect 
density  at  the  base  line  stress  level. 

The  process  of  applying  mechanical,  electrical  and/or 
thermal  stresses  to  an  equipment  item  for  the  purpose 
of  precipitating  latent  part  and  workmanship  defects 
to  early  failure. 

A  group  of  units  interconnected  or  assembled  to 
perform  some  overall  electronic  function  (e.g.> 
electronic  flight  control  '  system,  communications 

system). 

The  measurement  of  thermal  response  characteristics 
at  points  of  interest  within  an  equipment  when 
temperature  estremes  are  applied  to  the  equipment. 

A  self-contained  collection  of  parts  and/or 
assemblies  within  one  package  performing  a  specific 
function  or  group  of  functions,  and  removable  as  a 
single  package  from  an  operating  system  (i.e., 
autopilot  computer,  vhf  communicatiohs,  transmitter). 

The  masurement  of  vibration  response  characteristics 
at  points  of  interest  within  an  equipment  when 
vibration  escitation  is  applied  to  the  equipment. 

The  probability  that  an  equipment  will  pass  a  screen 
or  test  without  failure. 


3 . 2  Acronyms /Abbreviations 


3.2.1  Acronyms  use  in  procedure  8  of  section  5 


Description 

Airborne  Inhabited  Cargo 
Airborne  Inhabited  Trainer 
Airborne  Inhabited  Bomber 
Airborne  Inhabited  Attack 
Airborne  Inhabited  Fighter 
Airborne  Uninhabited  Cargo 
Airborne  Uninhabited  Trainer 
Airborne  Uninhabited  Bomber 
Airborne  Uninhabited  Attack 
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Aur 

ARW 

CL 

CB 

GT 

CM 

ML 

Mrr 

MTA 

MP 

MS 

HU 

Mtni 

MSB 

ira 

sr 

USL 


Othar  Acronyms 


Abbf  viatlon 

AOQL 

ATP 

BIT 

CMO 

COB 

DM 

ESO/EOS 

ESS 

.  FRACAS 
FL 

FMEA 

FBT 

IC 

ICT 

XCA 

LBS 

LRU 

LSI 

LTPD 

MTBF 

NLE 

MSI 

MFF 

OIN 

PB 

PEP 

PCS 

PPM 

PRVT 

PWA 

PM 


Airborn*  Uninhabited  Fighter 

Airborne  Rotary  Wing 

Cannon  Launch 

Ground  Benign 

Ground  Fixed 

Ground  Mobile 

Mlaaile  Launch 

Miaaile  Free  Flight 

Airbreathing  Miaaile  Flight 

Manpack 

Maval  Sheltered 

Maval  Unaheltered 

Maval  Under aea  Unaheltered 

Maval  Submarine 

Maval  Hydrofoil 

Space  Flight 

Undersea  Launch 


Deacr iotion 

Average  Outgoing  Quality  Limit 
Acceptance  Teat  Procedure 
Built  In  Teat 
Cannot  Duplicate 
Chance  Defective  Exponential 
Detection  Efficiency 

Electrostatic  Discharge/Electrical  Overstress 
Environmental  S'.ress  Screening 
Failure  Reporting  and  Corrective  Action  System 
Fault  Location 

Failure  Mode  A  Effect  Analysis 
Functional  Board  Tester 
Integrated  Circuit 
In  Circuit  Tester 
In  Circuit  Analyzer 
Loaded  Board  Shorts 
Line  Replaceable  Unit 
Large  Scale  Integration 
Lot  Tolerance  Percent  Defective 
Mean  Time  Between  Failures 
Maximum'  Likelihood  Estimate 
Medium  Scale  Integration 
Mo  Fault  Found 

Original  Equipment  Manufacturer 
Precipitation  Efficiency 
Production  Engineering  Phase 
Printed  Circuit  Board 
Parts  Per  Million 

Product  Reliability  Verification  Test 
Printed  Hiring  Assembly 
Performance  Monitoring 
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RTOK 

Rates t  OK 

SAF 

Stress  Adjustment  Factor 

SRU 

Shop  Replaceable  Unit 

SOL 

Specified  Quality  Level 

SPC 

Statistical  Process  Control 

TAAF 

Test  Analyse  a  Fix 

TQM 

Total  Quality  Management 
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4.  GENERAL  GUIDELINES 

4.1  Contractual  Aapecta  off  ESS.  ESS  must  camain  an  adaptive  process  so  that  the 
screening  regimen  can  be  changed  to  impro’'e  cost-effectiveness.  Contract 
provisions  for  ESS  programs  should  have  flesibility  to  effect  necessary 
modifications  of  stress  screens.  During  thS  initial  stages  of  production  more 
severe  stress  screens  may  be  required.  As  the  product  and  process  maturSf  the 
screens  may  require  adjustment  such  as  by  reducing  the  number  of  temperature 
eyclest  the  number  of  ases  of  vibration  or  by  eliminating  unnecessary  screens.  In 
early  production,  a  number  of  unknowns  preclude  adoption  of  optimum  stress 
screening.  Some  of  the  more  significant  unknowns  are! 

a.  Residual  design  deficiencies 

b.  Manufacturing  planning  errors 

c.  Worker  training 

d.  New  suppliers 

e.  Latent  defects  in  new  part  lots 

f.  .New,  process  capability 

g.  Precipitation  Efficiency 

h.  Detection  Efficiency 

The  stress  screening  program,  even  if  carefully  planned,  may  produce  unexpected, 
results  which  should  be  addressed  through  modification  of  the  screens.  The 
principle  of  adaptive  screening  is  to  adjust  the  screens  on  the  basis  of  observed 
screening  results  so  that  the  screens  are  always  most  cost  effective  while  meeting 
ISS  program  goals.  Contract  terms  should  be  flexible  enough  to  permit  modification 
of  screens  or  screen  parameters  when  such  modification  can  be  shown  to  be 
beneficial. 

In  long  term  production  the  quantity  and  distribution  of  latent  defects  changes 
with  time  and  therefore  contract  terms  should  contain  provisions  for  periodically 
reassessing  the  individual  screens  and  the  overall  ,  screening  program.  The 

overriding  criterion  for  change  should  be  the  most  cost  effective  achievement  of 
objectives.  Contracting  arrangements  should  be  made  which  permit  such  changes 
without  having  to  resort  to  extensive  renegotiation. 

4.2  Relation  of  ESS  to  MIL-STD-783  Reliability  Program  Tasks.  Planning  an  ESS 

program  for  the  production  phase  is  interrelated  with  many  of  the  MIL-STD-785 
reliability  program  tasks  which  are  required  to  be  performed  during  development  and 
production.  Every  effort  should  be  made  to  integrate  the  knowledge  gained  from 

MIL-STD-785  tasks  into  the  planning  of  an  ESS  program  for  production.  MIL-STD-/85 
reliability  program  tasks  which  have  a  particular  bearing  on  ESS  planning  include 
Reliability  Prediction  (Task  203),  Reliability  Allocation  (Task  202),  Qualificatio ^ 
Tests  (Task  303),  Parts  Program  (Task  207),  Pailura  Reporting  Analysis  and 

Corrective  Action  System  (Task  104),  Failure  Modes,  Effects  and  Criticality 
Analysis  (Task  204),  Reliability  Growth  Testing  (Task  302)^  and  of  course,  ESS 
(Task  301).  Proper  screen  selection  and  placement  is  highly  dependent  on  the 
reliability  and  stress  design  characteristics  of  the  equipment.  Information 
derived  from  reliability  program  tasks  such  as  predicted  and  demonstrated  failure 
rates,  quality  level  of  parts,  number  and  type  of  nonstandard  and  MIL-parts,  number 
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and  .typ«  of  Interconnections,  design  capab^ity,  field  stress  environments,  and 
critical  items  should  be  used  in  structuring' an  ESS  program  for  production. 

4.3  Subcontractor  and  Supplier  Stress  Screening.  Items  which  are  furnished  by 

subcontractors  or  other  equipment  suppliers  may  require  stress  screening.  There 

are  several  distinct  advantages  for  the  subcontractor  or  supplier  to  perform  the 
stress  screening  rather  than  the  prime  contractor. 

a.  Subeontractor/suppliar  concern  for  yield  can  be  translated  to 
profits  which  may  force  process  improvements  to  minimize  latent 
defects. 

b.  Screening  at  receiving  inspection/test,  by  the  prime  contractor, 

may  involve  returning  defective  items  to  the 
subcontractor/supplier  and  result  in  shortages  and  schedule 
slippages.  Performing  the  additional  screen  can  introduce  latent 
defects  due  ' to  handling  eg.  mechanical  and  ESD  '  damage  and 

electrical  over  stress. 

c.  Special  stress  screening  facilities  and  test  equipment  do  not 
have  to  be  purchased,  supported  and  operated  by  t^e  prime 
contractor. 

The  procedures  and  methodology  contained  in  the  Bandbook  can  be  imposed  on  the 
subcontractor/supplier.  To  assure  that  the  subcontractor/supplier  is  able  to 
perform  the  tasks  required  by  the  Bandbook  the  intent  must  be  made  known  prior  to 
production.  In  this  manner,  the  subeontracter/supplier ~can  prepare  a  screening 
plan,  acquire  the  necessary  capability  or  arrange  for  an  external  laboratory  to 
perform  the  screening. 

4.3.1  Screening  of  Soares.  Spares  should  be  subjected  to  a  screening  regimen 

equivalent  to  that  used  for  the  production  hardware.  Spares  are  either 

manufactured  oa  the  same  production  line  or  are  produced  separately  to  the  same 
specifications  as  the  production  hardware.  The  spares  are  most  often  an  LRU  or  SRU 
and  consequently  may  not  receive  the  exposure  to  additional  screening  at  higher 
assembly  levels  that  non-spare  items  might  receive.  Quantitative  ESS  goals  for  the 
system  should  be  allocated  down  to  the  spare  item.  The  procedures  of  Section  5  can 
be  used  to  ensure  that  defect  density  for  the  spares  does  not  exceed  allocated 
goals.  A  costly  and  less  desirable  alternative  would  be  to  screen  and'  test  all 
spares  in  a  mock-up  configuration  for  the  system. 

4.4  Planning  a  Stress  Serbening  Program.  Planning  a  stress  screening  program  must 
begin  early  in  the  design  phase  to  ensure  that  the  equipment  can  withstand  the 
necessary  ESS  stress  levels.  The  success  of  a  stress  screening  program  is  strongly 
dependent  on  knowledge  of  the  product  and  the  processes  to  be  used  in  manufacture. 
The  following  must  be  kept  in  mind  when  planning  a  stress  screening  program  using 
quantitative  methods: 

a.  The  defects  which  can  potentially  reside  in  the  product  and  the 
effectiveness  of  screens  in  precipitating  the  defects  to  failure 
(and  then  detecting  them)  Are  not  known  with  certainty.  By 
comparing  planned  estimates  for  defect  fallout  with  actual  screen 
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Calloutf  the  screening  end  manufacturing  process  can  be  adapted  to 
achieve  desired  goals. 

b.  Bsperience  data  on  equipment  similar  in  comp'^icion,  construction 
and  degree  of  maturity,,  can  provide  very  useful  data  for  planning 
purposes.  Information  derived  from  the  following  sources  should 
be  used  in  planning  an  BSS  program  for  production:. 

(1)  Identification  of  hardware  items  (parts,  assemblies)  which 
have  exhibited  a  high  incidence  of  latent  defectives  on  other 
programs. 

(2)  Identification  of  suppliers/vendors  whose  products  have 
indicated  high  defect  levels. 

(3)  Qualification  test  results. 

(4)  Supplier  acceptance  test  results. 

(5)  Part  receiving  inspection,  test  and  screening  results. 

(6)  Screening  and  test  records  for  previous  programs. 

(7)  Reliability  growth  test  results. 

(8)  Field  failure  data. 

e.  A  viable  screening  program  must  be  dynamic,  i.e.  the  screening 
process  must  be  continuously  monitored  to  ensure  that  it  is  both 
technically  and  cost  effective.  Changes  to  the  screening  process 
should  be  made,  as  necessary,  based  on  analysis  of  screening 
fallout  data  and  failure  analysis  so  that  quantitative  screening 
objectives  can  be  achieved. 

d.  The  basic  questions  which  must  be  addressed  in  planning  a  stress 
screening  program  are: 

(1)  Nhat  are  the  quantitative  objectives  of  the  programs? 

(2)  What  are  the  stress  screens  to  be  used  and  at  what  level  of 

assembly  should  the  screens  be  placed  to  achieve  the  desired 

objectives? 

(3)  What  are  the  costs  associated  with  each  of  the  possible 

alternative  screening  sequences  and  how  can  the  screening 
program  be  made  cost  effective? 

(4)  How  will  one  know  if  the  screening  program  is  proceeding 

according  to  plan?  Nhat  assurances  can  be  provided  that 
program  objectives  have  been  achieved? 
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(3)  What  correctlva  actions  must  ba  taken  to  achieve  desired 
screening  program  goals  i£  the  screening  fallout  data  indicate 
significant  departures  from  the  planned  program? 

e.  An  ESS  program  for  the  production  phase  sh'^uld  include  the 

following  major  tasks: 

(1)  Preparation  of  ESS  Plan 

(2)  Establish  Objectives/Coals 

(3)  Obtain  Planning  Estimates  of  Defect  Density 

(4}  Selection  and  Placement  of  Screens  to  optimise  cost 

A  discussion  of  each  of  these  major  tasks  which  includes  background, 
rationale  and  general  guidelines  for  use  of  the  detailed  procedures  is 
contained  in  4.4.1  through  4l4.S. 

4.4.1'  Preparation  of  ESS  Plans.  The  contractor  should  prepare  ESS  plans  for  both 
the  development  and  production  phases.  The  purpose  of  the  development  phase  plan 
is  to  describe  the. proposed  application  of  ESS  during  development  and  production 
and  to  validate  and  refine  the  estimated  values  of  Dxm  SS.  Use  of  the 

procedures  contained  in  the  Handbook  in  conjunction  with  stress  screen 
experimentation  on  pre-production  prototype  equipment  can  provide  invaluable  data 
for  planning.  Estimates  of  the  type  and  quantity  of  defects  likely  to  be  present 
in  the  hardware  can  b*  evaluated  against  experimental  data.  Screens  can  be 
designed,  based  upon  engineering  evaluation,  which  provide  the  desired  stress 
stimulation  for  suspected  defect  sites  in  the  hardware.  Test  specifications  can 
also  be  evaluated  to  ensure  that  possible  failure  nodes,  arising  from  various 
defect  types  and  sources,  can  be  detected  by  the  tests  performed  either  during  or 
following  the  screens.  Integration,  of  the  results  from  the  NIL-STD-785  reliability 
program  tasks  can  also  be  effectively  accomplished.  Early  fallout  from  screens 
provides  the  maximum  amount  of  information  on  likely  defect  sources  and  process 
capability.  Corrective  actions  taken  as  a  result  cf  screen  experimentation  during 
development  can  aid  significantly  in  stabilising  the  process  for  production.  The 
development  phase  and  production  phase  ESS  plans  should  be  submitted  for  approval 
by  the  procuring  activity  prior  to  production. 

4. 4. 1.1  Developmeint  Phase  Plan.  The  development  phase  plan  should  include  the 
following: 

a.  Identification  of  the  reliability  requirements;  for  the 
product  and  the  quantitative  goals  for  the  ESS  program. 

b.  Identification  of  the  equipment  to  be  screened  and  the 
respective  production  quantities. 

c.  Description  of  the  initial  screens  which  will  be  applied  and 
the  screening  experiments  which  will  be  conducted. 

d.  Description  of  the  data  collection  and  analysis  program  which 
will  be  used. 


;  N. 


4-4 


D00-a08K-344  (USAY) 


•.  D«scripCion  of  subcontraccor  and  supplier  stress  screening  to 
be  performed. 

f.  Results  of  preliminary  use  of  the  handbook  procedures. 

g.  Identification  of  the  organisation  elements  that  will  be 
responsible  for  ESS  planning  and  experimentation,  and  the 
conduct  of  development  phase  screening  activity. 

4. 4.1; 2  Production  Phase  Plan.  The  production  phase  plan  shall  include  the 
following:  , 


a.  Quantitative  objectives  of  the  ESS  program. 

b.  Detailed  breakdown  to  the  assembly  level  of  the  equipment 
which  will  be  screened. 

c.  Description  of  the  screens  which  will  be  applied,  including 
screen  parameters  and  exposure  time. 

d.  Description  of  the  results  in  applying  Procedures  A  through  E 
of  Section  S  including  the  rationale  for  achieving  quantitative 
objectives  in  a  cost  effective  manner. 

0.  Description  of  the  FRACAS  and  the  analyses  procedures  which 
will  be  used  to  evaluate  and  control  the  screening  process. 

f.  Description  of  the  PRVT  to  be  performed  to  verify 
achievement  of  objectives. 

g.  Identification  of  the  organisational  elements  responsible 
for  conducting  and  evaluating  the  effectiveness  of  the 
production  ESS  program. 


4.4.2  Establishing  Obiectives/Coals.  Expressed  quantitatively,  the  objective  of 
a  stress  screening  program  is  to  reduce  the  incoming  latent  defect  density  in  a 
production  lot  of  equipment  to  an  acceptable  remaining  latent  defect  density  in  a 
cqst  effective  manner.  Equipments  having  high  reliability  requirements  will  have 
more  stringent  goals  on  remaining  defect  density.  Methods  for  determining  goals  on 
remaining  defect  density  are  discussed  in  Appendix  A.  The  remaining  latent  and 
patent  defects  determine  the  field  reliability  according  to  the  following 
expression: 


Average  Failure  Rate  in  Field 


Total  failures  in  time  T 
_ 


=  MTBF 


■  suamation  of  (Opat  >  Olat.SAF. [l-expl -kT) ] 4-  CFR.T}/T  for  all  environments 


where: 


Dpat  "  remaining  patent  defects 
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Olftt  •  ttiMinin^  latent  daCacta 
SAt  ■  Straat  Adjuitment  Factor 
k  •  pcacipitation  strata  constant 

Meta  that  tha  DM  taraa  ara  abaant  slnca  tha  field  OE  is  1  by  definition. 

OsiRM  this  ralationahip,  tha  required  field  failure  rata  can  bo  used  to  dataraina 
tho  requiraaonta  for  roaukininq  dafaet  density  and  eonaaquantly  uaod  to  oatabliah 
qeala  and  raquiraaMnta  for  ail  intaqration  and  teat  laaala  fron  incoaiinq  defect 
donsitiaa  for  parts  throuqh  to  final  aquipamnt  taatinq. 

ha  axaapla  rolatinq  various  values  of  Oranaininq  to  tha  field  MTBF  is  shown  in 
Tablo  4.1  for  an  assumed  field  precipitation  rata  k*l/1000  Hr. 


Table  4.1  Ranaininq  Oafact  Density  Goals  (DR) 


Failure  Rate 
(Failures/Rours ) 

MTBF 

dremaiming 

(At  Field  Stress) 

0.009S16 

105 

10 

0.000991 

1091 

1 

0.00047S 

2102 

0.9 

0.000190 

9254 

0.2 

0.000099 

10908 

0.1 

0.000fl47_ 

21017 

0.09 

0.000019 

92942 

0.02 

0.C00Q09 

109083 

0.01 

O.OQOQOO 

1090833 

0.001 

4.4.3  Obtaining  Planning  Eatiaatas  of  Defect  Density.  The  dasiqn  of  a  stress 
aeraoninq  proqram  requires  knowladqe  of  the  quantity  and  typo  of  latent  defects 
which  are  likely  to  reside  in  tho  hardware  prior  to  assembly  level  screeninq.  The 
defect  density  tables  contained  in  Procedure  H  of  Section  S  are  used  to  obtain 
planninq  estimates  of  defect  density.  Values  in  the  tables  are  based  upon  studies 
of  historical  defect  data  from  tho  factory  and  field  for  several  part  types. 
Extrapolations  to  other  part  types  and  field  environments  were  made  based  upon 
correlations  '  to  NIt.-HDBK-217  quality  level  and  field  environment  factors.  Study 
results  and  methodology  are  contained  in  RaDC-TR-B6-149 .  Procedure  D  provides  the 
methodology  that  allows  the  user  to  refine  these  estimates  based  on  experience 
data. 

4. 4. 3.1  Latent  va  Patent  Defects.  A  common  understanding  of  tha  nature  of  the 
defects  which  tho  screening  program  should  be  designed  to  precipitate  is  essential 
for  proper  planning.  The  factors  which  impact  incoming  defect  density  and  tho 
rationale  for  the  procedures  used  in  obtaining  planning  estimates  of  defect  density 
should  also  bo  understood. 

For  ESS  purposes  defects  can  be  categorized  into  two  types. latent  and  patent.  A 
latent  defect  is  characterized  as  an  inherent  or  induced  weakness  or  flaw  with  some 
residual  strength  and  will  manifest  itself  as  a  failure  at  some  time  in  the  future 
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whan  CKpoasd  to  atrots  (oloctrlcalfmochaniral,  or  chtmical).  Latont  daCactt  can 
not  bo  dotoctod  until  procipltatod  aa  a  patant  dofact.  For  ainplicity,a  dofoct  with 
no  rtaidual  atcongth  but  requiring  atroaa  concurrent  with  toatinq  to  bo  dotoctablo 
can  alao  bo  conaidorod  to  bo  a  latent  defect  until  it  ia  detected.  Some  oxamplea 
of  latent  defecta  are: 

(1)  Parts 

(a)  P'^rtial  daaiaqo  through  electrical  owerstreaa  or 
eleetroatatie  discharge 

(b)  Partial  physical  daaago  during  handling 

(c)  Material  or  process  induced  hidden  flaws 

(d)  Baawge  inflicted  during  soldering  operations  (excessive 
heat) 

(2)  Interconnections 

(a)  Cold  solder  joint 

(b)  Inadequate/excessive  solder 

(c)  Broken  wire  strands 
.  (d)  Insulation  damage 

(o)  Loose  screw  termination 

(f)  Improper  crimp 

(g)  dnscated  connector  contact 

(h)  Cracked  etch 

(i)  Peer  contact  termination 

(j)  Inadequate  wire  stress  relief 

A  patert  defect  ia  a  defect  that  ia.  detectable  in  its  present  form  and  has  two 
subcategories,  error  and  precipitated  latent.  An  error  is  a  defect  caused  by 
workmanship  or  test  correlation.  Errors  are  preventable  and  should  not  occur, 
wheraas  patent  defecta  due  to  precipitated  latent  defects  are  only  preventable  to 
the  limits  of  the  state  of  the  art  in  equipment  and  technology.  Errors  can  be 
readily  monitored  using  conventional  SPC  techniques  and  can  be  removed  by  simple 
testing  or  inspection  without  the  need  for  ESS  or  environmental  stress. 

Errors  are  introduced  into  the  product  during  fabrication,  and  assembly,  and  pass 
through  various  assembly  stages  until  they  are  detected  by  a  test  or  inspection  of 
sufficient  thoroughness  and  subsequently  eliminated  from  the  product.  When  good 
quality  control  test  and  inspection  procedures  are  applied,  all  but  the  most  subtle 
errors  should  be  detected  and  eliminated  prior  to  shipment.  Some  examples  of 
errors  are: 

(1)  Parts 

(a)  Broken  or  damaged  in  handling 

(b)  Wrong  part  installed 

(e)  Correct  part  installed'  incorrectly. 

(d)  Missing  parts 

(e)  Blactrical  test  correlation  and  tolbrancing 

(2)  Interconnections 

(a)  Incorrect  wire  termination 

(b)  Opsn  wire  due  to  handling  damage 
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(e)  Wlra  thort  to  9round  duo  to  oiarouting  or  Inaulation 
dofflogo 

(d)  Miasing  wiro 

(o)  Opon  atch  on  printed  airing  board 

(C)  Opon  platod  ■>  through  hole 

(g)  .  Shortod  oteh 

(h)  Soidor  bridge 

(1)  Looao  wire  atrand 

h  proeipitatod  latent  defect  ia  a  latent  Clae  that  haa  been  tranaCoraMd ,  into  a 
patent  defect  by  eapoaure  to  atraaa  over  tine.  Since  detection  efficiency  ia  not 
100t.-aoaM  precipitated  latent  flaws«  and  errora,  will  eacape  to  the  field  aa 
undhtected  defecta.  It  ia  thua  iaiportant  to  addreaa  the  aapecta  of  precipitation 
and  detection  aeparately,and  alao  to  diatinguiah  and  aeparately  aonitor  error a  and 
precipitated  latent  flawa.  For  aiaplicity  the  Handbook  ahall  uae  the  term  patent 
defect  to  define  a  precipitated  latent  defect. 

4. 4. 3. 2  Categories  of  Defects.  The  majority  of  parts  and  connections  within  an 
electronic  equipment  will  never  fail  over  the  product's  lifetime  and  ace  thus 
"good*.  The  failures  which  occur  during  product  life  ace  traceable  to  design  or 
eatecnally  induced  eauaea>  or  to  latent  defects  which  were  introduced  into  the 
product  during  aanufacture.  Such  dofeetSf  if  not  eliainated  from  the  product  in 
the  factory,  will  result  in  preaature  or  early-life  failurea  in  the  field.  Rot  all 
latent  defecta  however,  .are  acreenable  i.e.,  capablo  of  being  eliainated  from  the 
equipaent  in  the  factory  by  uae  of  etreae  screens.  It  ia  only  those  latent 
defects,  whose  failure  threshold  can  b«  accelerated  by  the  stresses  imposed  by  the 
screens,  which  are  acreenable.  It  ia  the  acreenable  early  life  failure  which  the 
streaa  screening  prograa*  Rust  be  designed  to  remove.  Figure  4.1  illustrates  the 
categories  of  defects  and  their  relationship  to, product  life  failures. 

4. 4. 3. 2.1  Screenable  Latent  Oefects  and  the  Field  Stress  Hnvironaent.  The  notion  of 
acreenable  latent  defects  must  be  further  eaaained  to  fully  understand  the 
rationale  used  for  the  procedures  contained  in  the  handbook.  The  population  of 
latent  defects  within  newly  manufactured  electronic  items  can  be  viewed  as  a 
continuusi  which  ranges  from  minor  defects  of  small  size  to  major  defects  of  large 
size. 

However,  it  ia  important  to  note  a  somewhat  controversial  point,  i.e.,  given  the 
same  manufacturing  process,  the  number  of  latent  defects  which  may  reside  in  the 
hardware  will  differ,  depending  upon  the  operating  environment  and  stress  levels  to 
which  the  equipment  will  be  t’xposed.  The  stress/time  to  which  a  latent  defect  is 
exposed  will  determine  its  failure  threshold  and  time-to-failure.  The  probability 
of  a  latent  defect's  failure  threshold  being  exceeded  is  much  higher  in  a  harsh' 
environment  than  in  a  more  benign  environment. 

Obtaining  an  initial  estimate  of  defect  density  for  an  equipment  must  take  into 
consideration  the  Sijld  operating  environment  to  which  the  equi^ent  will  be 
exposed  during  product  life. 

Since  the  operating  environmental  stress  levels  are  different  and  less  than  the 
factory  ESS  levels,  the  field  defect  density  estimate  is  not  directly  applicable  to 
the  factory  EUS  program.  Further,  the  producer  must  design,  assess,  and  monitor 
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tlw  BS  pro^ru  based  upon  analysis  of  factory  fallout  data  and  causes.  Some 
■etbod  aust  thus  be  provided  to  relate  defect  density  in  the  field  to  the  factory 
defect  density.  This  is  accoaplished  by  including  a  stress  adjustaent  factor  (SAf) 
in  the  aedel,  where 


STRESS  ADJUSTMEMT  FACTOR  = 


DEFECT  DENSITY  (FIELD  STRESS) 
DEFECT  DENSITY  (FACTORY  STRESS) 


The  application  and  aeasureaent  of  the  SAT  is  described  in  Procedures  B  and  B 
respectively  of  section  9. 

4. 4. 3. 3  factors  Which  laoact  Defect  Oensitv.  The  quantity  and  type  of  defects 
which  are  introduced  into  a  product  are  dependent  upon  several  factors.  The  first 
sis  factors*  listed  below,  are  related  to  product  or  prograa  characteristics  for 
which  the  aanufacturing  function  within  a  ccapany  has  little  control.  The  last  two 
factors  are  related  to  the  aanufacturing  process  for  which  the  manufacturing 
function  has  direct  control. 


4-9 


DOO-aDex-344  (USAT) 


4.  Complexity  -  Th«  quantity  and  typo  of  parts  and 

intorconnections  uSod  In  the  product  affects  defect  density. 
Increased  complexity  creates  more  opportunities  for  defects. 

b.  Part  Quality  Lavel/Grada  -  The  quality  levels  of  parts  are 
established  by  M1L>STD  part  screening  requirements.  The 
number  of  defects  which  remain  in  a  lot  of  screened  parts  is 
determined  by  the  type  and  extent  of  screening  and  tasting  to 
which  the  parts  are  subjected  under  NIL~STO  screening 
requirements. 

c.  Stress  Invironment  -  The  stress  conditions  to  which  the 
equipment  will  be  exposed  will  affect  the  proportion  of 
defects  which  should  be  screened  from  the  product.  A  defect 
may  be  precipitated  to  early  failure  in  a  harsh  field 
operating  environment,  but  may  survive  product  life  in  a 
benign  field  environment. 

d.  Process  Maturity,  -  Mew  procjction  requires  time  to  identify 
and  correct  planning  and  process  problems,  train  personnel 
and  to  establish  vendor  a:td  procass  controls.  Maturity  is 
dependent  on  volume  and  tii:e.  Low  production  volume  over  a 
long  period  would  have  a  low  maturity  rate  and  will  thus 
impact  defect  density. 

e.  Packaging  Density  •  electronic  assemblies  with  high  part 
and  .  wiring  density  are  more  susceptible  to  process, 
workmanship  and  tempe/ature  induced  defects  due  to  smaller 
error  margins,  increased  rework  difficulty  and  thermal 
control  problems. 

f.  Concurrent  engineer ing  Proper  design  analysis  and 
assessment  and  application  of  Concurrent  engineering 
principles  during  the  design  stage  will  tend  to  ensure  a 
reliable  and  producible  product  and  thus  reduce  the  latent 
(and  error)  defect  densities.  Durability  analyses  will  also 
ensure  that  the  design  can  withstand  the  stresses  of  BSS. 

The  following  factors  are  under  the  direct  control  of  the  manufacturing  function. 
The  degree  of  control  exercised  will  determine  defect  density.  Screen  fallout  data 
provide  the  necessary  input  for  determining  out-of-control  conditions. 

g.  Manufacturing  Process  Controls  -  Good  process  controls  will 
tend  to  reduce  the  number  of  defects  which  arc  introduced 
into  the  product. .  The  criteria  by  which  processes  are 
considered  to  be  in  or  out  of  control  should  be  established 
by  reliability  requirements  and  monitored  using  the  fallout 
from  the  screening  process. 

h.  Norkmanship  Quality  Standards  -  Stringent  and  properly 
enforced  workmanship  quality  standards  will  enhance  the 
reliability  of  the  product  through  reduced  introduction  of 
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nockmnship  daCacts  into  tha  product.  Tha  lav-ils  to  which 
quality  standards  . should  ba  astablished  and  monitorad  must 
also  bo  dictated  by  reliability  requirements  and  made  visible 
by  the  screening  process. 

4*4.3. 3.1  Part  vs  Assembly  Degeet  Density.  Tha  part  defect  density  can  have  a 
significant  iapact  on  the  assesibly  defect  density  depending  upon  the  number  of 
parts  contained  in  tha  assesibly.  The  Poisson  approaiaation  is  used  in  Pigure  4.2 
to  illustrate  tha  eapected  assembly  defect  density  as  a  function  of  the  remaining 
part  defect  density  and  tha  number  of  parts  per  assembly.  As  can  be  noted 
relatively  small  values  of  part  defeqt  density  result  in  large  values  of  assembly 


Pigure  4.2  Fraction  of  Defective  Assambli.i^  vs  Remaining  Part  Fraction  Defective 
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dcfact  dansity  dapending  upon  the  number  o£  parts  contained  in  the  assembly.  As  an 
example,  for  a  ISO  part  assembly  containing  parts  with  a  Defect  density  of  .01 
(10,000  PPM),  the  assembly  defect  density  is  l.S.  In  terms  of  yield,  only  about 
22%  i.e.  exp(-l.S)  of  such  assemblies,  when  subjected  to  first  assembly  test,  would 
pass  without  failure.  It  is  quite  obvious  that  the  part  defect  density  must  be 
much  better  than  .01  if  the  costs  of  rework,  retesting  and  handling  of  the 
asaeablies  are  to  be  avoided.  The  questions  answered  by  the  ISS  methodology  and 
procedures  in  this  handbook  are:  How  much  better  must  the  remaining  part  defect 
density  be?;  What  level  of  part  defect  density  is  needed  for  delivered  systems? 
and,  can  such  levels  be  achieved? 

4. 4. 3. 3. 2  Part  Level  vs.  Assembly  Level  Screening.  Screening  at  the  part  level 
may  be  a  cost  effective  alternative  for  eliminating  defects  prior  to  the  parts 
being  assembled  into  the  production  hardware.  A  population  of  parts,  even  those 
procured  to  high  quality  levels,  may  appear  to  contain  high  defect  density  levels. 
For  example,  microelectronic  devices  procured  to  the  quality  requirements  of  MIL- 
STO-883  receive  1Q0%  final  electrical  testing  by  the  part  vendor.  Nonetheless,  one 
manufacturer  has  found  that  about  1%,  and  as  much  as  4%  of  the  parts  will  not  pass 
a  similar  electrical  test  performed  at  the  OEM  receiving  inspection.  There  are 
several  possible  reasons  for  this  including: 

the  seller's  and  buyer's  tests  are  different 
seller  testing  errors 

.  buyer  testing  errors 

.  device  damage  or  degradation  in  handling 

inspection  and  sorting  errors, 
latent  defects 

General  awareness  of  this  problem  in  the  industry  has  resulted  in  improvements  in 
part  quality  and  reliability  and  the  results  may  no  longer  be  applicable.  For 
example,  results  reported  in  the  Integrated  Circuit  Screening  Report  published  by 
the  IIS  in  November,  1988  indicated  a  significant  improvement  for  microcircuits  and 
revealed  that  the  additional  handling  involved  in  the  .rescreening  process  was 
actually  introducing  more  defects  that  were  being  screened. 

Hone  the  less,  it  should  be  noted  that  the  foregoing  discussion  addresses  errors 
only  and  must  be  extended  to  include  latent  defects  and  that  it  is  primarily  latent 
defects  that  escape  to  the  field  and  degrade  early  life  reliability. 

The  requirement  for  parts  rescreening  should  not  be  mandated  and  should  only  be 
used  as  determined  to  be  necessary  by  the  implementation  of  the  Handoook. 

Screening  at  the  assembly  level  is  also  a  means  of  finding  and  eliminating  part 
defects  from  the  hardware.  The  part  fallout  from  early  screening  at  the  assembly 
level  can  provide  much  of  the  information  needed  for  resolving  such,  uncertainties 
and  taking  corrective  action.  There  are  always  uncertainties  as  to  whether  the 
part  defects  which  arc  found  during  assembly  level  screening,  are  escapes  from  part 
level  screens  or  whether  they  are  newly  introduced  defects  due  to  handling,  test 
and  assembly  operations.  A  thorough  failure  analysis  of  the  fallout  from  assembly 
level  screening  can  help  in  determining  defect  causes  and  the  types  of  screens 
which  should  be  used. 
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4. 4. 3. 3. 3  Air  PorcB  RSM  2000  ESS  Policy-Part  Fraction  Oefectivg.  Air  Force  R&M 
2000  ESS  studies  reconunend  that  the  manutactur ing  process  begin  with  piece  parts 
having  a  remaining  part  fraction  defective  below  1000  PPM  by  FYS?  and  below  100  pPM 
by  FT90.  Procedure  D  of  Section  S  and  ESS  results  are  used  in  the  .  Handbook 
procedures  to  evaluate  the  achievement  of  thuse  goals.  However,  the  prescribed 
requirement  of  100  PPM  defect  level  for  parts  may  not  be  adequate  for  achieving  the 
required  reliability.  The  actual  requirements  should  be  determined  using  Procedure 
A  and  My  increase  or  relax  the  RSM  2000  levels.  The  RSM  itdOO  levels  should  also  be 
interpreted  as  being  applicable  to  both  latent  and  parent  defects  where  the  patent 
defects  include  errors,  due  to  electrical  testing,  test  correlation,  specification 
discrepancies  etc. 

4. 4. 3. 3. 4  Process  Maturity  and  Defects.  The  maturity  of  both  the  product  design 
and  the  Mnufacturing  process  can  significantly  impact  the  quantity  and  type  of 
defects  which  can  reside  in  the  hardware.  The  data  shown  in  Table  4.2  represent 
experience  on  several  large  development  and  production  projects.  As  the  data 
illustrate,  the  proportions  of  failures  in  a  product  which  are  traceable  to  design, 
part  or  manufacturing  causes  can  differ  substantially,  depending  upon  the  stage  of 
maturity  of  the  product  and  the  manufacturing  process.  During  the  development 
phase,  the  major  contributor  to  product  failure  is  design  (50%),  while  parts  may 
account  for  20!l  of  the  failures.  Unfortunately,  design  problems  can  still  be 
present  in  the  product  when  stress  screens  are  being  conducted  during  production. 

The  proportion  of  failures  in  a  product,  attributable  to  design,  would  be  expected 
to  decrease  as  the  process  Mtures.  The  overall  defect  density  in  the  product 
would  also'  be  expected  to  decrease  as  the  process  matures.  Maturity  of  the  product 
and  process  should  be  taken  into  account  when  planning  estimates  of  defect  density 
are  being  determined  in  accordance  with  Procedure  B  of  Section  5.  In  such  cases, 
the  user  may  decide  to  use  Procedure  0  to  modify  the  defect  density  values  in 
Tables  5.2  through  5.13,  of  Procedure  B  either  upward  or  downward,  depending  upon 
past  experience  and  assessments  of  maturity.  With  an  emphasis  on  TQM  and 
concurrent  engineering,  more  thorough  design  analysis  and  assessment  should  be 
performed  during  the  design  stage  to  prevent  design  problems  during  production.  A 
high  incidence  of  design  problems  during  initial  production  provides  valuable 
feedback  on  the  efficacy  of  the  concurrent  engineering  program. 


Table  4.2  Defect  Types  a  Density  vs  Process  Maturity 


Maturity 

Defect  Type  Distribution  (percent) 

Defect  Density 

Design 

Manufacturing 

Parts 

Development 

40-60 

20-40 

10-30 

High 

Early  Production 

20-40 

30-50 

20-40 

Moderate 

Late  Production 

5-15 

20-30 

60-70 

Low 

4. 4. 3. 3. 5  Packaging  Density.  Assemblies  with  high  part  and  wiring  density  are 
more  likely  to  contain'  both  patent  (error)  and  latent  defects  because  of  the 
proximity  of  devices  and  interconnections  contained  within  a  small  volume.  The 
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«££«et>  o£  poor  hoot  dissipation  in  donsoly  packaged  electronic  assemblies  can 
accelerate  latent  de£scts  to  early  £ailure.  Oi££iculties  in  initially  assembling 
or  reworking  the  hardware  can  also  make  such  assemblies  more  defect  prone. 

Procedure  B  in  Section  S,  £or  estimating  defect  density,  thus  includes  a  packaging 
density  factor.  This  factor  should  be  continually  monitored  and  refined  using 
Procedure  0  of  section  S. 

4.4.4  Screen  Selection  and  Placement.  Planning  a  stress  screening  program 
reguires  the  selectionand  placement  of  appropriate  screens  at  various  levels  of 
assembly  so  as  to  achieve  a  cost  effective  screening  program.  Listed  below  are  the 
factors  which  affect  screen  selection  and  placement.  The  factors  axe  discussed  in 
more  detail  in  the  following  paragraphs. 

a.  Screening  strength  -  The  product  of.  precipitation  efficiendy  and 

detection,  efficiency,  determines  the  capability  for  removing 

defects. 

b.  Precipitation  efficiency  -  Prior  knowledge  of  the  effectiveness  of 
the  screens  in  precipitating  defects  to  failure. 

e.  Detection  efficiency  -  The  tests  which  can  be  economically  and 

feasibly  used  to  detect  defects  which  have  been  precipitated  to 
failure  by  the  screens. 

d.  Thermal  and  vibration  response  characteristics  -  The  structural, 
thermal  and  material  properties  of  the  items  to  be  screened  and 
their  response  to  applied  stress. 

e.  Design  limits  -  The  environmental  stress  design  limits  of  the 
items  to  be  screened. 

f.  Facilities  -  The  screening,  test  and  instrumentation  facilities 

available  to  the  manufacturer  '  to  perform  screening  and  test 
operations. 

g.  Costs  •  The  costs  to  achieve  screening  program  goals  on  remaining 
defect  density. 

i.  Product  Reliability  Verification  Test  (PRVT),  -  The  use  of  a  PRVT 
as  an  integral  part  of  an  KSS  program  to  provide  confidence  fipld 
reliability  will  be  achieved. 

4. 4. 4.1  Precipitation  Efficiency.  Precipitation  efficiency  is  defined  as  the 
probability  that  a  screen  will  precipitate  a  defect  to  a  detectable  state  given 
that  a  defect  susceptible  to  the  screen  stress  is  present.  Screening  strength  is 
defined  as  the  precipitation  efficiency  multiplied  by  the  probability  that  the 
defect  will  be  detected  and  removed  (i.e.,  the  detection  efficiency).  A  basic 
premise  of  stress  screening  is  that  under  specific  screening  stresses  applied  over 
time,  the  failure  rates  of  defectives  are  accelerated  from  that  which,  would  occur 
under  normal  field  operating  stress  conditions.  By  subjecting  electronic  items  to 
accelerated  stresses,  i.e.  rapid  temperature  cycling  and  random  vibration,  latent 
defects  are  thus  precipitated  to  early  failure.  More  severe  stresses  will  tend  to 
accelerate  failure  mechanisms  and  the  rate  of  defect  failure.  For  example,  the 
failure  rate  of  a  latent  defect  increases  with  more  rapid  rates  of  temperature 
change  and  larger  temperature  extremes.  The  precipitation  efficiency  (and  hence 
screening  strength)  of  a  random  vibration  screen  increases  as  a  function  of  the 
level  and  duration  of  the  applied  excitation. 
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Strssa  scraans  ara  not  all  aqually  affaetiva  in  tranaforming  latant  dafacts  into 
datactabla  failucaa.  Tabla  4.3  providaa  a  liating  of  latant  defect  types  and  the 
acreana  baliavad  to  ba  affactiva  la  praeipitating-  tham  to  failuce.  Tabla  4.3  may 
ba  uaad  aa  an  aid  in  tha  aalaetion  of  a  acraan  typa  whan  prior  knowledge  on 
workmanahip  or  part  dafacta  for  aimilar  aaaambliea  ia  not  available. 


Tabla  4.3  Aaaambly  bafact  Typea  Precipitated  by  Thermal  a  Vibration  Screens 


Defect  Type 

Thermal  Screen 

Vibration  Screen 

Defective  Part 

X 

X 

Broken  Part 

X 

X 

Improperly  Installed  Part 

X 

X 

Solder  Connection 

X 

X 

PCS  etch.  Shorts  and  Opens 

X 

X 

Loose  contact 

X 

Mira  Insulation 

X 

Loose  wire  termination 

X 

X 

Improper  crimp  or  mating 

X 

Contamination 

X 

Debris 

X 

Loose  hardware 

X 

Chafed, pinched  wires 

X 

X 

Hermetic  seal  failure 

X 

Adjacent  boards/parts  shorting 

X 

Raferanca  RAOC-TR-82-87 


Table  4.3  indicates  that  vibration  screens  ara  generally  more  effective  for  loose 
contacts r  debris  and  loose  hardware  while  temperature  cycling  screens  are  not 
effective.  Thermal  screens  are  generally  more  effective  for  part  parameter  drift, 
contamination  and  improper  crimp, or  mating  typa  dafacts  while  vibration  screens  are 
not.  for  other  defect  classes  listed  in  tha  tabla,  both  thermal  and  vibration 
screens  ara  affactiva,  but  tha  relative  degree  of  of factivenasa  of  one  screen  type 
over  tha  other  is  not  precisely  known.  These  ara  soma  of  tha  uncertainties  which 
must  ba  dealt  with  in  planning  a  screening  program.  Historically,  on  average,  20% 
of  tha  defects  ara  found  to  bo  responsive  to  vibration  screens  and  80%  to 
tamparatura  cycling  screens.  (Rafaranca  publication  lES  Bnvirci.nmantal  Stress 
Screening  Guidelinos  for  Aaaambliea). 

To  improve  tha  modelling  accuracy  and  to  ensure  a  proper  balance  between  thermal 
and '  vibration  screens,  it  is  recommended  that  the  defect  population  be  segregated 
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into  RV  sonsitivo  daCacts  and  TC  sanaitiva  dafacts.  If  nacassary,  tha  population 
roaponsiva  to  aithar  TC  or  RV  can  alao  bo  includad  on  tha  modal. 

4. 4.4. 1.1  Scraan  Paraieatara.  Praeipitation  afficiancy  ia  a  function  of  apacific 
acraan  atroaaaa  (paranatora)  and  tha  tima  duration  of  tha  atroaa  application. 
Xquationa  providad  in  Procadura  C  of  Saction  S  provida  valuaa  for  praeipitation 
afficiancy  as  a  function  of  ralavant  acraaninv  paraaatara.  It  should  bo  notad  that 
thosa  paraawtara  partain  to  tha  unit  undor  tost  and  not  tha  chambar  ate. 

Vibrational  charactaristies  of  tha  aquipnant  (aq.  raaonancoa,  transmiasibility 
ate.)  and  tha  various  thortaal  eonductivitios  and  aassoa  must  ba  considorad.  All 
assomblod  hardwaro  consists  of  many  paths  along  which  a  strass  might  ba 

transnitiad.  Tha  salaction  of  scroaning  paraaatara  and  mathods  of  strass 
application  must  ba  auitad  to  tha  strass  transmission  charactaristies  of  tha 
hardwaro  dasign.  As  a  part  of  tha  scraan  salaction  and  placamant  procass,  in  which 
'thormal  or  vibration  scroans  ara  to  ba  usod,  a  strass  rasponsa  survay  of  tha  itam 
to  ba  seraanad  should  ba  parforvad.  This  may  raquira  siirulations  and  or  survoys 
conductad  on  tha  actual  or  similar  hardwaro.  Cara  should  ba  axarcised  to  ensura 
that  hardwara  rasponsas  ara  larga  enough  to  ganarata  an  effective  screen  while  not 
exceeding  hardwara  design .  capability .  Environmental  stresses  should  be  applied  to 
tha  hardwara  and  tha  response  of  critical  hardwara  alemants  measured  to  determine 
whether  maximum  or  minimum  tamparatura  limits  ara  being  axceadad>  and  whether 
suspected  defect  aitaa  (parts,  intarconnaetions  ate.)  ara  rasponsiva  to  tha  screen 

stress.  In  addition,  normal  design  provisions  for  isolating  tha  hardware  from 

stress  such  as  tha  use  of  shock  mounting,  vibration  isolators  or  cooling  air 
should  also  ba  evaluated.  Application  of  anvironmantal  strass  screening  in  such 
instances,  should  raquira  bypassing  tha  normal  strass  isolation  provisions  or  may 
dictate  tha  need  for  screening  at  lower  assembly  levels  which  do  not  include  tha 
stress  isolation  design  features.  Temperature  cycle,  constant  temperature,  random 
and  swapt-sina  screening  paramatars  ara  defined  as  follows: 

a.  Thermal  evcla  scraan  oaramatars 

(1)  Maximum  tamparatura  (Tux)  ~  The  maximum  tamparaturp  to  which  tha 
seraanad  itam  will  ba  exposed.  This  should  not  exceed  the  lowest 
of  tha  maximum  ratings  of  all  tha  parts  and  matarials  comprising 
the  item.  Rota  that  non-oparating  temperature  ratings  for  parts 
are  higher  than  operating  ratings. 

(2)  Minimum  tamparatura  (Tmin)  ~  minimum  temperature  to  which  tha 
screened  item  will  ba  exposed.  This  should  not  exceed  the  highest 
of  tha  minimum  ratings  of  all  tha  parts  and  materials  comprising 
tha  assembly. 

(3)  Range  (R)  -  Tha  range  is  the  difference  between  tha  maximum  and 
minimum  applied  axtarn>il  (chambar)  temperature.  (Tnax  ~  Tmin)* 
Tamparaturas  ara  axprassad  in  *C. 

(4)  Tamparatura  rata  of  change  (T)  -  This  parameter  is  tha 

average  rate  of  change  of  tha  temperature  of  tha  item  to  ba 
screened  as  it  transitions  between  Tmax  sod  Tmin  is  given 
by: 
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Nh«r«:  ti  is  th«  transition  tima  Croa  Tain  to  Taos  i"  ninutos 

t2  is  ths  transition  tiaia  froa  7aax  to  Tain  in  ainutss 

(5)  pwoll  -  Maintaining  tha  hardwara  tomparatura  constant,  onca 
it  has  raachad  tha  aaaiaua  (or  ainiaum)  tanparature,  is 
rafarrad  to  as  dwall.  Tha  duration  of  tha  dwell  is  a 
function  of  diftaraacas  in  tha  tharnal  nass  of  tha  itams 
baing  tcraanad. 

(6)  Nuabar  of  cycles  -  The  nunber  of  transitions  between 
temperature  extremes  (Tmax  or  Tmin)  divided  by  two. 

b.  Constant  Temperature  Screen  Parameiters 

(1)  Temparatura  delta  (  AT)'-  Tha  absolute  value  of  the 

diffaranea  between  the  hardware  temperature  and  25°C. 

A  T  -  It  -  25  ‘Cl 

Nhara  T  is  t.he  hardware  temperature 

(2)  Duration  -  Tha  tima  period  ever  which  the  temperature  is 

applied  to  tha  item  baing  screened,  in  hours,  after  the 

'  hardware  has  reached  thermal  equilibrium. 

c.  Vibration  Screen  Paramatars 

(1)  Gras  level  for  random  vibration  -  Tha  rms  value  of  tha 

applied  power  spectral  density  observed  by  tha  hardware, 
including  rasonanca  and  transmissibility  effects. 

(2)  6-lavel  for  swept  sine  vibration  -  The  constant  rms 
acceleration  applied  to  the  equipment  being  screened 
throughout  the  frequency  range  above  40HZ.  The  g-level  below 
40HZ  may  be  less, 

(3)  .  Duration  -  Tha  tima  period  over  which  the  vibration 

excitation  is  applied  to  tha  item  being  screened,  in  minutes. 

(4)  Axas  of  vibration  -  This  can  be  a  single  axes  or  multiple 
axes  depending  on  the  sensitivity  of  defects  to  particular 
axial  inputs. 

4. 4. 3. 1.2  Design  Limits.  The  use  of  screen  parameters  which  impose  stresses  which 

exceed  the  design  limits  o^  the  product  is  not  recommended.  Effective  screening 
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progruit  can  ba  4av«Lop«d  without  having  to  raaort  to  itraasaa  which  axcawd  tha 
dasiqn  capability  of  tha  hardwata.  Critaria  for  judging  how  much  tha  dasign  limit* 
can  be  safely  eaceadad,  without  causing  damage  to  tha  product,  are  non-axiiStant  or 
at  least  arbitrary.  Rowavar,  to  permit  reasonably  high  ESS  strass  levels,  it  is 
important  that  tha  equipment  be  designed  for  ESS  and  thus  tha  ESS  program  and 
raquirad  stress  levels  should  ba  datarminad  concurrently  during  tha  design  stage. 
Using  tha  procedures  contained  in  tha  handbook,  tha  manufacturer  can  focus  on  those 
items  in  which  dafacts  are  most  likely  to  reside  in  tha  hardware  and  determine  safe 
screening  levels,  within  appropriate  cost  constraints,  for  precipitating  them  to 
failure.  The  procedures  taka  into  account  the  increased  defects  with  increased 
factory  Stress  level  and  also  require  a  fatigue  life  study  to  ensure  that  useful 
operating  life  has  not  been  iapact,ed  by  the  amount  or  level  of  ESS. 

4. 4. 4. 1.3  Guidelines  for  Initial  Screen  Selection  and  Placement.  The  development 
phase  ESS  program  is  intended  to  expose  various  defect  types  and  causes  an(^  to 
obtain  factory  data  to  calculate  and  refine  the  planning  estimates  of  Dm  and  SS 
that  were  based  on  handbook  and  industry  data.  Additional  ESS  beyond  that  intended 
for  production  may  be  required  to  improve  the  estimate  accuracy.  An  initial 
screening  regimen  should  be  selected  for  experimental  use  during  the  development 
phase  in  conjunction  with  the.  use  of  the  handbook  procedures.  Table  4.4  is 
recommended  as  an  aid  in  selecting  and  placing  screens  for  a  starting  regimen. 

4. 4. 4. 1.4  RaM  2000  ESS  Initial  Regimen.  RaM  2000  ESS  studies  recommend  the  screen 
types,  parameters  and  placements  outlined  in  Table  4.S  as  an  initial  regimen.  The 
screens  contained  in  Table  4.S  have  high  precipitation  efficiency.  After 
sufficient  fallout  has  been  observed,  the  screening  regimen  nay  be  reduced. The  RaM 
2000  guidelines  thus  represent  initial  values  fer  consideration  during  the 
development  phase  and  can  be  reduced  for  production  based  on  the  planning  and 
analysis  procedures  outlined  in  Procedures  A  and  D. 

4. 4. 4. 2  Detection  Efficiency.  Detection  efficiency  is  a  measure  'f  the  ability  to 
detect  and  remove  patent  defects.  Detection  efficiency  includes  factors 
representing  fault  coverage,  the  requirement  for  concurrent  stress,  the  test 
duration,  and  the  diagnostics  and  rework  capabilities  for  removing  'the  defect. 
Detection  efficiency  is  expressed  as  the  ratio  of  patent  defects  detected  (and 
removed)  by  A  defined  test  procedure  to  the  total  possible  number  of  patent 
defects.'  While  stress  screens  may  be  effective  in  precipitating  a  latent  defect 
into  a  detectable  failure,  removal  of  the  failed  condition  is  dependent  on  the 
capability  of  the  test  procedures  used  to  detect  and  localize  the  failure. 

Care  should  be  taken  to  ensure  thac  tests  have  detection  efficiencies  as  high  as  is 
technically'  and  economically  achievable.  The  screens  may  otherwise  precipitate 
defects  to  failure  which  may  go  undetected  by  post  screen  tests.  Modern  electronic 
equipment  comprised  of  microprocessors,  large  memory  and  LSI  devices  may  contain 
defects  so  subtle  that  only  the  most  thorough  of  tests  can  detect  them.  High 
screening  strengths  at  lower  levels  of  assembly  may  not  always  be  easily 
accomplished  because  of  low  detection  efficiency.  The  difficulty  in  accurately 
simulating  functional  interfaces  or  the  inability  to  establish  meaningful 
acceptance  criteria  may  make  the  development  of  tests  with  high  detection 
efficiency  at  the  assembly  level  difficult  and  costly.  A  certain  percentage  or 
defects  may  oniy  be  detectable  at  the  unit/system  level  when  ail  or  a  majority  of 
the  system  components  are  connected  and  operating  as  a  system.  Analysis  and 
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quantification  ef  dataction  affieianeiat  should  ba  an  intagral  pact  of  tha  planning 
for  a  secaaning  pcogcaa. 

Tabla  4.4  Guidalinaa  for  Initial  Sercan  Salaction  and  Placamant 


Satactlaa 

U«a) 

af  Taao  Const,  land 
Astaibly  Cjrclt  Tons.  VIS. 


Plaeanant 

Mrantagas 

Olsadvantagas 

C  •  Cffaet1*a 
N  ■  NtrglAsIly  CffKtIva 
N  •  ibt  Cffsetiva 

•bttt: 

1.  fsnleulorly  \f  poatr 
It  lOoHod  and  parfor* 
aanea  ft  aonitorsd  at 
tanparatura  aitraatt. 

Z.  Cffaetfva  aiiara 
ttsaaaMat  eontafa  ca*> 
Slat  davleat  (IUm.  a<en» 
proeattari,  Syanfdt) 

3.  Cffaetivanass  sigsly 
daaawaawt  aa  assaaafy 
ttractura.  Not  aVfaetlaa 
for  mil.  stiff  NAt. 


taaast  (unaoMarad 
teraaat) 

a  Saail  tin  paraits 

Sates  leraaaing 
a  Loa  tsaraal  aasi 
allom  Sigs  ratas 
af  ttivaratura 
ehaaga 

a  Taaairatiira  ranga 
graatar  tsaa  apar* 
atlag  raaga  I 

tllaaabla 


laittivaly  aaty  ta 
(mar  and  «altor 
parfaraanct  during 
Kraan 

Nlgsar  tast  datac> 
tlaa  afficlaney 
than  assanly 
laaal 

dSMualy  Intar* 
eannactlaas  (a.g.. 
Hiring  Saekplana) 
art  icraanad 


Thaml  aatt  pra* 
eludas  sign  ratas 
of  esanga,  or 
raaulres  costly 
faellltiat 
Cost  par  flaw 
significantly  - 
SIgnar  than 
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raducad  fro* 
atsaasly  laval  ■ 


Difficult  and 
eattly  ta  tast  at 
t—aratura 

lltlHPil 

Hass  pracludas  uta 
af  affaetlaa 
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or  Mkat  Hsa 
eastly 

Cast  par  flap  Is 
Sigfeast 
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4. 4. 4. 3.1  D«f twining  Dwfetlon  Ettlel^ncv.  Owtaction  affieiancy  ii  datacmincd  as 
tha  product  of  factors  that  raprasant  tha  following  conaidarations: 

(a)  Tl.a  pcooabillty  of  obsarving  and  datacting  a  patant  dafect. 
This  includas  tha  probability  of  dataction  and  tha 
probability  of  occurrahca.  Considaration  must  also  ba  givan 
to  tha  aatant  that  tha  tests  and  liaits  being  used  reprasant 
all  application  raguiraaants  for  functional  and  parametric 
parfornanca.  Tha  detection  of  intaraittant  and/or  situation 
sansitiva  defects  aay  also  require  aatandad  test  times  and 
may  ba  aodallad  using  a  Poisson  distribution. 

(b)  .  Tha  raquiraaant  for  concurrent  stress.  Many  of  tha  latent 

flaws  pracipitatad  to  failure  by  ESS  can  only  ba  detected 
whan  stress  is  applied  during  tha  test. 

(c)  Tha  probability  of  isolating  and  than  removing  the  dafect 
without  creating  an  additional  dnfact. 


Table  4.S  RSF  000  Environmental  Stress  Screening  Initial  Regimen 


SCREEN  TYPE  PAft.‘ METERS  AMD 
CONDITIONS 

ASSEMBLIES 
(PRIMTED  MIRINGS 
ASSEMBLIES)  (SRU) 

EQUIPMENT,  OR 

UNIT  (LRU/LRM) 

THERMAL  CYCLING  SCREEN 

Tamparatura  Range 
(Miniaua)  (sea  Rota  1) 

Proa  -34 ‘C 

To  ♦OS’C 

Proa  -54*C 

TO  ♦?1*C 

Tamparatura  Rata  of  Change 
(Miniaum)  (sea  Mote  2) 

30*C/Minute 
(Chamber  Air  Tamp) 

5*C/Minuta 
(Chambar  Air  Temp) 

Tamparatura  Dwell 

Duration  (See  Note  3) 

Until 

Stabilisation 

Until 

Stabilisation 

Tamparatura  Cycles  (Minimum) 
Power  On/Equipmant  Operating 

23 

No 

10 

(See  Rota  S) 

Equipaant  Monitoring 

No 

(See  Rote  6) 

Elactrical  Tasting  At'tar  Screen 

Yes 

(At  Ambient  Temp) 

Yes  (At  Ambient  Tamp) 

QUAS-RANOOM  VIBRATION 
(Sea  Rota  7) 

Spectral  Density 

(Sea  Note  8) 

6  Grms 

Frequency  Limits 

100-1000  HZ 

Axes  Stimulated  Serially 
or  concurrently 

3 

Duration  of  Vibration 
(Minisnia) 

-Axes  stiaulated  serially 
-Axes  stimulated  concurrently 

10  Ninutas/Ax.is 

10  Minutes 

Power  On/Equipmant  Operation 

(See  Rota  5) 

Equipment  Monitoring 

(See  Note  6) 
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On  soaw  syitsm  procurtmnnts  thn  probability  b£  detaction  is  a  specified  parameter 
for  built-in-test  (BIT),  performance  monitoring  (PM)  and  fault  location  (PL) 
capability  requirements.  When  the  required  BIT  or  PM/FL  capability  is  used  to 
verify  performance  of  an  item  being  screened,  the  actual  values  of  fault  coverage 
should  be  used  in  conjunction  with  the  factors  defined  above  and  in  Procedure  C. 
On  other  system  procuremerts,  requirements  to  perform  a  failure  modes  and  effects 
analysis  (FMEA)  are  specified  in  the  contract.  In  such  cases,  the  FMEA  should  be 
used  to  estimate  the  fault  coverage  for  a  given  test  design. 

When  FMEA  or  BIT  fault  detection  requirements  are  not  specified  in  the  contract, 
estimates  of  fault  coverage  should  be  made  based  upon  experience  datb.  Appendix  C 
provides  values  of  fault  coverage  for  various  tests  which  may  be  applied  with 
stress  screens.  The  values  in  the  table  were  derived  by  production  and  engineering 
test  personnel  from  a  large  000  electronic  system  manufacturer..  RACC  TR-82-87 


4. 4. 3. 2. 2  Power-On  Testing  vS  Power-Off.  Application  of  power,  exercising  and 
monitoring  equipment  performance  continuously  during  the  screen  will  greatly 
enhance  detection  efficiency.  Subtle  faults,  such  as  contact  intermittents  or 
temperature  sensitive  parts,  can  only  be  detected  with  powered  and  monitored 
screens.  With  the  increased  complexity  of  modern  electronics,  fault  sites  may  be 
confined  to  smaller  areas  and  fault  symptoms  may  appear  only  during  certain  tests 
or  under  a  special  set  of  external  conditions.  As  a  result,  a  greater  incidence  of 
"Cannot  Duplicate”(CIiO) ,  "Mo-F^ult  Found”  (NFF)  and  "Retest  OX"(RTOK)  and  similar 
intermittent  or  transient  phenomena  can  occur.  Patent  defects  which  have  been 
precipitated  to  failure  by  stress  screens  can  be  categorised  into  three  general 
types:  _  _ 

a.  Type  1  Physical  defects  transformed  from  an  inherent  weakness  to  a 

hard  failure  by  the  stress  screen. 

b.  Type  2  Physical  defects  that  manifest  as  failures  only  while  under 

thermal  or  mechanical  stress,  (e.g.  intermittent  caused  by  a  cold 
solder  joint). 

c.  Type  3  Functional  defects  that  manifest  as  performance  failures  or 

anomalies  only  while  under  thermal  or  mechanical  stress,  (e.g.  timing 
problems ) . 

The  type  1  defects  arc  readily  detected  by  post  screen  tests  of  sufficient 
thoroughness.  type  2  and  Type  3  defects  require  thorough  and  continuously 
monitored  tests  so  that  they  can  be  detected.  Type  3  defects,  which  include 
problems  such  as  timing,  part  parameter  drift  with  temperature  or  tolerance  build¬ 
up  can  only  be  detected  with  powered  and  mbnitored  tests.  Type  2  and  Type  3 
defects  can  comprise  504  and  as  much  as  804  of  the  latent  defects  present  in  the 
hardware.  (Reference  RAOC  TR-86-149) 

Developing  tests  and  test  strategies  for  use  with  stress  screens  and  estimating 
their  detection  efficiency  is  a  vitally  important  activity  in  planning  a  stress 
screening  program.  The  use  of  tests  with  high  detection  efficiency  is  of  equal 
importance  to  using  screens  with  high  precipitation  efficiency  in  structuring  a 
screening  program  for  production. 
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4. 4. 4. 3. 3  Pr«/Pd«t  Scfn  Testing  and  Scr««nlno  Strength.  In  ord«r  to 

■xporiMntally' dotocmino  acrooninq  vtronqth,  th*  Collowing  conditiona  ac«  required:, 

a.  Tha  Itaaa  aubjaetad  to  atraaa  actaaninq  auat  ba  taatad  thoroughly 
baCoca  tha  atraaa  aeraan  to  aaaura  that  no  datactabla  failures 
caawin  at  the  atart  of  atraaa  aeraaning.  Vttien  taating  ia  not 
parforawd  prior  to  strass  aeraaning,  it  ia  not  known  whothar 
patent  dafaeta  ware  praaant,  which  could  have  baan  detected 
without  atraaa  aeraaning,  or  whothar  latebt  dafaeta  ware 
praelpltatad  by  the  atress  aeraan. 

b.  Tha  itena  aubjaetad  to  atraaa  aeraaning  must  ba  powerad  and 
exareisad.  Parfornanea  must  bo  continuously  monitored  to  assure 
that  straaa-^dapandant  defects  (a.g.,  intarmittartts,  tamparatura 
and  timing  sensitive  faults)  are  dataeted. 

e.  Tha  items  aubjaetad  to  screening  must  ba  tested  using  the  same 

tast(s)  both  before  and  after  tha  strass  screen  to  assure  that 
tha  failures  datactad  are  a  result  of  th*  stresses  imposed. 

d.  Data  auat  ba  collactad  on  defect  fallout  after  tha  strass  screen 
(i.a.  ,  during  subsequent  strass  scraans,  tests,  or  early  field 
operation)  to  obtain  an  aatimata  of  tha  number  of  defects  which 
ware  initially  praaant. 

Nhon  such  data  are  available  and  assuming  perfect  teats,  then  the  screening 
strength  can  ba  datorminad  by  use  of  the  observed  fdllout  from  the  screen  and  the 
number  of  defects'  initially  present  i,c.: 


Screening  Strength  • 


Fallout 


Mumber  of  Initial  Latent  Defects 


However,  the  total  number  df  latent  defects  can  not  be  determined  until  extensive 
field  data  ia  available.  We  are  thus  compalled  to  use  a  modeling  approach  where 
screening  strength  is  based  upon  estimates  derived  from  a  combination  of  the  actual 
screening  program  data,  axparimants,  and  tha  published  literature.  The 
precipitation  efficiency  models  and  values  used  in  the  handbook  tables  'of  Procedure 
C  in  Section  S,  were  developed  using  such  an  approach.  The  results  and  methodology 
used  for  these  studies  are  contained  in  RADC  TR-82-87  and  RAOC  TR-86-149. 
Additional  information  is  also  provided  in  AFWAL  TR-80-3086  and  ADR  14-04-73.  As 
more  exper fence  data  on  stress  screening  are  gathered,  the  screening  strength 
estimates  will  be. refined  and  improved. 

4. 4. 4. 2. 4  Pre  and  Post  Screen  Testino  Ouiino  Production.  Performing  tests 
before,  during,  and  after  environmental  exposure,  as  discussed  in  4. 4. 4.2. 3,  may  ba 
useful  during  tha  dayalopmant  phase  but  represents  an  undesirable  non-value  added 
expanse  during  production.  An  alternate  method  is  required.  The  analysis 
mathodology  provided  in  Procedure  D  is  based  upon  curvefitting  actual  data  to 
determine  tha  latent  and  patent  defect  components.  Defects  present  Qefore 
screening  appear  as  the  Dp  term  and  defects  precipitated  and  detected  by  the  screen 
appear  as  the  Dt,  term.  This  approach,  however,  requires  a  sufficient  number  of 
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data  points  throughout  tho  screen.  If  changes  take  place  during  production  such  as 
in  an  assembly  or  fabrication  process,  personnel  or  production  flow,  then  the 
defect  density  (both  latent  and  patent)  is  likely  to  change  and  affect  the  fallout 
observed  during  screening  and  will  be  apparent  using  the  monitoring  and  control 
procedures  of  Procedure  B.  Under  long  term  production,  process  improvements  and 
other  corrective  actions  taken  as  a  result  of  the  screening  process  are  likely  to 
change  the  quantity  and  distribution  of  latent  defects  present  in  the  hardware. 
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4.5  Production  Phaia  -  Monitoring  Evaluation  and  Control.  Onco  a  screaning 
program  is  inplamantaU  during  tha  production  phasa,  tha  scraan  Callout  data  and  tha 
scraaning  procass  must  ba  monitorad  and  controllad  to  assura  that  program 
objactivcs  ara  achiavad.  For  an  aCfactiva  monitoring  and  control  program,  tha 
Ciald  raliabllitp  raquiramants  should  ba  diractly  ralatad  to  goals  and  raquiramants 
Cor  parts,  procassas,  and  aatarials  and  assaablias  Cor  all  factory  intagration  and 
ISS  tast  lavtls.  Tha, procadura  for  astablishing  thasa  raquiramants  and  Cor  monitor 
and  control  ara  providad  in  Procaduras  A  and  I  raspaetivaly.  Usa  of  a  Pailura 
llopor;:in9  Analysis,  and  Corractiva  Action  Systam  (FRACAS)  should  be  an  integral 
part  of  production  phase  monitoring  and  control  tasks.  The  fallout  from  the 
scraaning  process  provides  tha  necessary  visibility  regarding  the  sources  of 
defects  in  tha  product  and  tha  manufacturing  process.  Finding  defects,  detacmining 
their  root  causes  and  ensuring  that  tha  sources  of  tha  defects  ara  eliminated  from 
either  the  process  or  product,  is  tha  basic  mechanism  by  which  process  capability 
is  improved. 

Analyses  of  screen  fallout  data  must  ba  performed  with  specific  objectives  in  mind. 
Mell'dafinad  monitoring,  aval  nation  and  control  task  objectives  will  ensure  that 
tha  proper  data  is  collected,  classified  and  corractly  analyzed  to  meet  objectives. 
The  objectives  of  the  monitoring-evaluation  and  control  tasks  are  to  establish 
assurance  that  remaining  defect  density  and  reliability  goals  ara  achieved  through 
iaplamenting  improvements  in  manufaeturing,  screening  and  tast  process  capability. 
Manufacturing  process  capability  is  improved  through  taking  corrective  actions 
which  reduce  the  number  of  defects  that  are  introduced  into  the  product.  , Screening 
process  capability  is  improved  by  increasing  both  the  precipitation  efficiency  of 
screens  (by  ensuring  that  potential  sites  for  defects  in  the  product  are  being 
adequately  stimulated)  and  the  detection  efficiency. 

Another  goal  of  monitoring  and  control  is  related  to  cost  effectiveness.  The 
initial  screening  program  might  have  been  based  upon  planning  estimates  which  were 
overly  pessimistic.  ■  Corrective  actions  night  also  have  been  taken  during 
production  to  reduce  the  ntimber  of  defects  introduced  into  the  product.  In  either 
case,  if  the  screening  program  is  continued  as  planned,  more  screening  than  is 
necessary  results,  which  impacts  both  cost  and  schedule.  Decisions  must  be  made  on 
how  to  reduce  the  screening  regimen.  In  a  sense,  the  goal  of  ESS  and  the  monitoring 
and  control  tasks  is  to  make  the  screening  program  unnecessary  (except  for  that 
limiting  value  required  for  PRVT). 

4.5.1  Data  Collection.  The  importance  of  timely  and  accurate  data  collection  to 
achieving  screening  program  objectives  cannot  be  overemphasized.  The  data  elements 
1  sted  below  should  be  collected  during  the  conduct  of  the  screening  program.  Some 
of  the  data  elements  become  available  directly  as  observed  events  from  .the 
screening  process.  Other  data  elements  will  become  available  only  after  analysis 
of  the  failures  and  failure  data,  or  after  a  batch  of  items  have  been  exposed  to 
screening. 

a.  Identification  of  the  items  exposed  to  the  screen/test,e.g. , 
description,  part  number,  revision,  and  serial  number, 

b.  Number  of  like  items  exposed  to  the  screen/test. 

c.  Number  of  like  items  passed/failed  the  screen/test. 
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d.  Data  of  tost 

•.  Toat  station  or  squivalant 


f.  Typa  and  numbar  of  dafacts  found  in  conjunction  with  tha  number 
of  itons  asposadt  pasaad/failad  (data  alamants  b,  c,  d). 

9.  Oaaeriptlon  of  tha  typa  of  dafact  found  (part, 
workaanship/procass,  dasi9n) 

h.  Idantif ication  of  tha  part,  intarconnaction  sita  where  tha 
defect  was  found. 

i.  Identification  of  tha  assambly  level  or  manufacturing  process 
operation  where  tha  defect  was  introduced. 

j.  Screen  conditions  under  which  tha  defect  was  found  (a. 9.,  high 
tenperatura,  vertical  axis  of  vibration  ate.}. 

k.  Tiaa-to-failura  relative  to  the  start  of  tha  screen. 

l.  Failure  analysis  results  which  identify  tha  root  causa  of  tha 
dafact. 

m.  Corrective  action  taken  to  eliminate  the  causa  of  tha  dafact 
from  tha  product  and/or  process. 

Data  alamants  1  and  m  may  only  be  available  if  trends,  as  identified  by  the  SPC 
monitoring  and  control  methodology,  warrant  detailed  root  causa  analysis  and 
corrective  action. 

4.S.2  Failure  Classification.  In  order  to  establish  a  basis  for  tha  analysis  of 
the  screening  fallout  data,  tha  failures  must  be  properly  classified.  The 
following  classification  scheme  is  recommended. 

a.  Fart  defect  -- A  failure  or  malfunction  which  is  attributable  to  a 
basic  weakness  or  flaw  in  a  part  (diode,  transis.or,  microcircuit, 
etc.)  Subcategories  may  include  electrical,  electronic,  and 
mechanical. 

b.  Manufacturing  defect  -  A  failure  or  malfunction  attributable  to 
workmanship  or  to  tha  manufacturing  process  (cold  solder  joint,  cracked 
etch,  broken  wire  strands,  etc.)  Subcategories  may  include  assembly, 
process,  and  handling. 

c.  Design  Failure  -  A  failure  or  malfunction  attr  -'  utable  to  a  design 
deficiency.  Rote  that  electrical  or  thermal  oversti  :ss  failures  due  to 
inadequate  derating,  are  design  problems.  Subeategor ias  include 
hardware  and  software. 
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d.  Extarnally  induced  Cniluces  -  A  failur*  attributable  to  external 
influences  such  as  prime  power  disturbances,  test  equipment, 
instrumentation  malfunctions  or  test  personnel. 

e.  Dependent  failure  -  A  failure  which  is  caused  by  the  failure  of 
-another  associated  item  which  failed  independently. 

f. .  Unknown  cause  failure  -  An  independent  failure  which  requires 
repeir  and  rework  but  which  cannot  be  classified  into  any  of  the  above 
cateqories.  An  intermittent  failure  that  recur r.s  infrequently  would  be 
an  unknown  cause.  Subeateqories  include  verified  and  not  verified. 

q.  Unable  to  verify  (UTV),  retest  ok  (RTOK),  and  MO  Fault  Found  (MFF) 
classifications  describe  conditions  where  an  anomaly  during  testing 
could  not  be  reproduced. 

4.5.3  Preliminary  Analysis  of  Fallout  Data.  A  preliminMry  analysis  of  the  fallout, 
data  should  be  performed  to  insure  that  failure  causes  are  properly  established  and 
to  categorise  the  failures  so  that  more  detailed  analysis  related  to  the  ESS 
program  objectives  can  be  performed. 

a.  All  failures  traceable  to  part  board  and  interconnection  defects, 
which  are  precipitated  and  dateeted  by  a  sereen/tast,  should  be  considered  to  be 
latent  defects  provided  that  pre-screen  testing  was  performed.  These  data  should 
be  used  for  monitor  and  control  purposes. 

b.  A  predominance  of  design  problems  which  are  discovered  during 
production  screening  operations  is  a  matter  of  serious  concern.  Every,  effort 
should  be  made  to  determine  corrective  actions  for  design  problems  very  early  in 
production.  It  does  no  good  to  speculate  that  the  design  problems  should  have  been 
eliminated  from  the  hardware  during  the  development  stage.  Stress  screening,  on  a 
loot  basis,  is  an  expensive  and  tine  consuming  method  for  finding  design  problems. 
If  the  fallout  from  screening  indicates  persistent  evidence  of  design  problems, 
methods  other  than  10d4  stress  screening  should  be  used.  Reliability  growth  and 
Tast-Analyse-And-Fix  (TAAF)  techniques  ace  recommended. 

c.  Special  attention  should  be  given  to  unknown,  cause  failures. 
Sufficient  investigation  should  be  made  to  establish  that  an  intermittent  condition 
does  not  exist.  The  number  of  failures  classified  as  "Unknown  Cause”  should  be 
kept  to  a  minimum.  Every  effort  should  be  made  to  correlate  the  failure 
circumstance  data  with  the  other  similar  failure  incidents,  as  well  as  to  use 
failure  analysis  so  as  to  establish  the  cause  of  failure.  The  number  of  "unknown 
cause*  classifications  and/or  "unable  to  verify”  classifications  should  be  used  in 
assessing  the  detection  efficiency. 

d.  Analyses  of  induced  failures  should  be  performed  to  determine 
necessary  corrective  actions. 

The  detailed  analyses  would  typically  be  performed  if  the  established  goals  and 
requirements  are  not  being  achieved,  either  for  parts,  materials  and  processes  or 
for  assemblies  at  various  ESS  levels. 
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4.5.4  Analvaia  of  Screen  Fallout  Data.  Tha  analyaia  oC  acraening  fallout  data  ia 
dicactad  toward  avaluating  the  acraening  proeeaa  ao  aa  to  achieve  acreening  program 
goala  on  remaining  defect  denaity  Dremaiming*  Yield  goala  are  achieved  by  both 
improving  manufacturing  proeeaa  capability  through  corrective  action  and  by 
improving  tha  acraening  and  teat  proeeaa  capability  when  it  ia  found  to  be  needed. 

Manufacturing,  acraaning  and  taat  proeeaa  capability  will  determine  the  remaining 
defect  denaity.  The  capability  of  theaa  procaaaea  are  meaaured  and  controlled  by 
uaa  of  two  important  quantitiea.  the  incoming  defect  denaity  (Out)  and  the 
acreening  atrength  (SS).  Neither  one  of  theae  guantitiea  are  directly  obaervable 
aa  a  reault  of  the  screening  proeeaa.  The  only  obaervable  atatiatic  is  the  fallout 
from  the  screen/test,  from  which  inferences  regarding  Dm  and  SS  must  be  drawn. 
The  basic  approach  used  in  Procedure  0  of  Section  5,  ia  to  obtain  estimates  of  Dxn 
and  SS,  using  the  screen  fallout  data  and  to  statistically  compare  the  observed 
data  against  the  planning  estimates.  Based  upon  the,  comparisons,  corrective 
actions  are  determined  to  eliminate  the  source  of  the  defect  from  the  process 
and/or  to  change  the  screens  so  aa  to  achieve  stated  objectives. 


Two  complementary  procedures  are  presented  in  Procedures  0  and  E  for  performing 
monitoring  and  analyses  tasks.  Procedure  0  uses  curve  fitting  techniques,  applied 
to  the  mathematical  model,  to  estimate  Oih  and  SS.  Procedure  E  uses  Quality 
Control  Charts  (SPC  and  PARETO)  for  monitoring  and  control. 


Quality  control  charts.  The  use  of  control  charts  for  defect  control  is  a  standard 
technique.  Control  charts  (SPC  and  PARETO)  are  used  in  Procedure  E  which  are  based 
upon  the  Poisson  Probability  distribution;  i.e.. 


P(«) 


e-D  D* 


s! 


Where: 


0 


a 

»(*) 


« 

m 


defect  density 

number  of  defects  in  an  item 
probability  of  x  defects  in  an  item 


The  mean  of  the  Poisson  distribution  is  0  and  the  standard  deviation  is  Vo.  The 
primary  purpose  of  the  control  chart  technique  is  to  establish  baselines  against 
which  the  process  can  be  monitored  and  by  which  out-of-control  conditions  can  be 
identified.  Because  of  varying  conditions,  for  example  improving  defect  density, 
the  actual  defect  density,  0  is  determined  using  regression  analysis.  This  value 
is  then  used  to  determine  the  expected  statistical  variation  due  to  limited  sample 
site 


i.e.  D  ±  n  V  — 
N 


where  n  is  the  number  of  standard  deviations,  typically  3,  and  N  is  the  sample  or 
lot  size. Defect  density  is  calculated,  using  the  fallout  data,  and  compared  against 
the  control  chart  baselines.  ^art  and  workmanship  (process)  problems  are  rank 
ordered  with  consideration  for  the  expected  defects  based  on  complexity  etc.  and 
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analysas  ar«  e«c£ocm«d  and  corractiva  actions  takan  to  aliminata  the  source  of  the 
defects  from  the  product.  Procedure  B  of  Section  5  contains  the  detailed 
aothodology  for  implementinq  the  control  chart  technique. 

4.5.4.1  Use  of  the  Mathematical  Model  to  Evaluate  Screening  Results.  Appendix  A 
provides  a 'description  of  the  Stress  Screening  Mathematical  Model.  The  factory 
fallout  data  (expressed  defects  per  system)  can  be  curve  fitted  to  the  expression 
developed  therein  (for  OpxMoVEO)  >0  a*  to  obtain  estimates  o*  the  model  parameters. 
Parameters  vhich  can  be  determined  using  this  method  are  Di)|>  SS  (comprising  PE  and 
DB  terms),  the  constant  failure  rate  (CER)  and  SAP,  a  stress  adjustment  factor 
relating  defect  levels  at  field  stress  to  factory  stress. 

4.5.4.2  Use  of  the  Chance  Oefective  Exponential  (COE)  Model  to  Evaluate  Screening 
Results.  The  defect  distribution  for  both  factory  and  field  stress  environments 
have  been  empirically  determined  to  be  represented  by  the  following  expression. 

Oremoved-OE.  (  Dp  *  01.(l-e*-kt)  CPR.t  ] 

where  Dp  represents  the  patent  defects, 01  represents  the  latent  defects,  t  the 
stress  duration  eg  tine, cycles  etc,  k  the  orecipitation  stress  constant,  CFR  the 
constant  failure  rate,  and  Oe  is  the  detection  efficiency  which  is  1  for  the  field. 

The  COS  model  developed  by  Pertig  and  Kuthy  and  discussed  in  a  paper  contained  in 
the  197E  Annual  RCM  Symposiur  provides  a  possible  explanation  for  this  observed 
relationship. 

Regardless  of  the  true  derivation,  the- empirical  results  have  been  found  to  be 
sufficiently  accurate  for  the  purposes  of  this  handbook.  Inaccuracies  either  in  the 
modelling  and  or  the  estimated  parameters  are  initially  addressed  using  design 
margins  and  addressed  during  the  production  phase  through  the  use  of  actual  factory 
and  field  data  to  refine  the  estimates.  The  observed  fallout  data  can  be  fitted  to 
the  model  to  obtain  estimates  of  the  model  parameters.  The  parameters  of  the  model 
provide  estimates  of  the  incoming  defect  density  DIM,  the  screening  strength  (SS, 
PS,  OS),  the  limiting  failure  rate  of  the  equipment  (CPR)  and  the  stress  adjustment 
factor  (SAP).  Pigure  4.3  is  an  extract  from  a  study  report  which  shows  a  histogram 
of  the  screen  fallout  from  a  12  cycle  -54*C  to  71*C  temperature  cycle  screen.  The 
fallout  per  cycle  is  used  to  obtain  maximum  likelihood  estimate  (MLE)  for  the 
parameters  of  the  CTE  model. 

As  Pigure  4.3  shows,  the  COE  m'del  parameters  estimated  by  the  MLE  procedure,  are; 
incoming  defect  density  (DIM)  equal  to  .1342  defects  per  item,  the  failure  rate  of 
a  defect  (Ok)  equal  to  .1483  failures  per  hour  (which  corresponds  to  a  screening, 
strength  of  .93)  and  a  value  of  .0032  for  the  limiting  failure  rate  (CFR). 

d.S-4.3  Product  Reliability  Verification  Test  (PRVTI.  The  use  of  a  PRVT  segment 
as  part  of  an  ESS  program  is  intended  to  provide  confidence  that  field  reliability 
will  be  achieved  and  help  identify  out  of  control  conditions  that  could  otherwise 
be  missed.  As  defect  density  is  improved,  ESS  can  be  reduced  to  optimize  cost 
without  impacting  field  reliability.  However,  ESS  can  not  be  completely  eliminated 
since  some  portion  is  required  to  allow  reliability  to  be  assessed.  PRVT  is  that 
portion  of  ESS  retained  for  this  purpose. 
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rigura  4.3  Taaparatura  Cycling  Data  rittad  to  tha  Chanea  Oafactlva  Bnponantial 

Nodal 

Asaassaants  of  roliability  should  ba  mada  on  tha  basis  of  tha  parforaanca  of  tha 
eollactiva  population.  Tha  PRVT  sagnant  should  ba  iaplamantad  on  a  first  pass 

yiald  basis  (first  pass  yiald  baing  dafinad  as  tha  nuabar  of  systems  completing  the 
PRW  segment  with  no  failuras  divided  by  tha  total  number  of  systems  submitted 
first  time).  ZS  tha  first  pass  yield  requirements  are  not  achieved,  corrective 

actions  must  bo  takan  that  address  tha  entire  population.  Appandia  B  provides  the 
mathematical  darivation  of  tha  PRVT  methods  contained  in  tha  handbook.  Procedure  P 
in  Section  S  contains  tha  detailed  procedures  for  incorporating  the  PRVT  segment. 

Rota  that  a  failure  free  raquiramant  for  any  part  of  SSS  or  PRVT  is  not 

rocoaswndad.  If  raquiramants  (eg.  PRVT  yiald)  are  not  baing  aehiavad  and  dafacts 
are  randomly  distributed,  than  tha  overall  defect  density  is  too  high  and  action 
must  ba  takan  that  affects  tha  entire  population.  Requiring  one  particular  piece 
of  equipment  to  pass  a  saquanca  of  tests  "failure  free”  does  not  substantially 

improve  tha  reliability  of  cha  papulation.  Tha  failed  item  however,  must  undergo 
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sufficient  confidsnes  testing  subsequent  to  cewotk  to  ensure  that  the  fault  has 
been  eliminated. 

4.6  Costs  of  ESS  vs  Produetivitv  Imorovement.  The  costs  of  conducting  a  screening 
program  during  the  production  phase  can  be  high.  To  a  large  extent,  the  costs  can 
be  offset  by  the  increased  productivity  vhich  results  through  proper  screen 
selection  and  placement.  Screening  at  the  lowest  possible  level  of  assembly  will 
almost  always  be  the  least  costly  alternative  in  terms  of  rework  costs.  The  time 
and  effort  required  to  test,  troubleshoot  and  repair  items  Increases  by  at  least  an 
order  of  magnitude  at  each  subsequent  level  of  assembly.  Significant  cost  savings 
or  avoidance  can  accrue  to  the  manufacturer  by  analysing  the  cost  benefits  of 
various  screen  selection  and  placement  alternatives  and  by  striving  to  find  defects 
at  the  lowest  possible  level  of  assembly.  The  fixed  and  recurring  costs  to  screen, 
instrument  and  test  the  hardware  at  lower  assembly  levels,  especially  with  power 
applied,  can  possibly,  negate  any  benefit  from  lower  rework  costs  thus,  the  optimum 
ESS  program  must  be  determined  for  each  equipment  type.  .Cost  savings  to  the 
Government  will  result  through  improved  field  reliability  and  corresponding 
reductions  in  field  repair  costs.  The  benefits  of  a  properly  conducted  ESS  program 
to  the  Government  go  beyond  field  repair  costs  alone.  Improved  reliability  during 
early  life  will  also  reduce  over-buying  of  spares,  since  estimates  of  required 
spare  quantities  are  based  upon  early  life  field  performance.  The  opportunity  for 
introducing  new  defect  sources  into  the  hardware  during  field  maintenance  and 
handling  is  also  reduced. 

There  should  be  however,  controls  and  constraints  on  the  cost  of  conducting  a 
screening  program.  Situations  can  arise  where  the  cost  of  conducting  a  screening 
program  far  outweigh  any  benefits  which  may  be  derived.  Ter  -example,  for  low. 
complexity  items  the  number  of  screenable  defects  which  are  likely  to  be  present  in 
t.he  hardware  may  be  relatively  small.  Conducting  a  full-scale  screening  program; 
in  such  cases,  can  result  in  very  high  costs  per  defect  eliminated.  Costs  of  $10K 
to  $1SE  per  defect  eliminated  may  be  justified  for  equipments  which  are  used  in 
critical  missions  with  very  high  reliability  requirements.  On  the  other  hand,  such 
costs  may  be  difficult  to  justify  if  the  equipment  is  used  in  noncritical  missions 
and  if  the  costs  of  field  maintenance  are  not  severely  effected  by  not  screening. 
Each  case,  where  a  stress  screening  program  is  under  consideration,  must  be  judged 
individually  as  to  the  cost  benefits  to  be  derived  from  stress  screening  and 
optimized  on  a  combined  user-producer  cost  basis.  Procedure  A,  in  Section  5  is 
used  to  determine  the  cost  effectiveness  of  ESS  programs. 

4.6.1  Pacilities  and  Costs.  The  facilities  that  the  manufacturer  has  available 
for  screening,  instrumenting  and  testing  the  product  affects  screen  selection  and 
placement.  A  manufacturer  may  not  have  random  vibration  facilities  or  automatic 
test  systems  which  can  be  used  for  the  stress  screening  program.  In  such  cases, 
the  manufacturer  may  decide  to  impose  less  severe  stresses  for  a  longer  duration  or 
decide  to  use  less  bxpensive  alternatives  such  as  described  in  RAVMAT  P-9492.  The 
costs  to  purchase  expensive  screening  or  test  equipment  and  perform  screens  at  a 
given  level  of  assembly  may  not  be  warranted,  in  terms  of  the  number  of  defects 
which  are  likely  to  be  found.  The  screening  and  test  facilities  which  the 
manufacturer  has  available  for  screening  must  be  addressed  in  preparing  the 
screening  program  plan  and  in  the  screen  selection  and  placemert  process.  Costs 
versus  the  benefits  to  be  derived  from  screening  should  be  addresrii’d. 


4-30 


OOO-aOBIC-344  (OSAF) 


Tha  critarion  usad  in  tha  handbook  to  judga  cost  affactlvanass  is  the  combined  cost 
to  the  producer  and  customer.  If  tha  cost  i>av  defect  eliminated  is  found  to  be 
higher  than  required  or  optimum,  then  the  manufacturer  should  determine  alternative 
methods  which  lower  the  costs  of  finding  and  eliminating  the  defects.  Alternatives 
night  include  reducing  the  incoming  defect  density  by  means  other  than  ass'tmbly 
screening,  te.g.,  increase  the  quality  level  of  parts  used)  increase  the  screening 
Strength  at  lower  assembly  levels,  or  eliminate  screens  which  may  be  of 
questionable  value.  In  those  cases,  where  field  reliability  is  an  overriding 
requirement,  then  the  Government  procuring  activity  must  decide  6n  the  appropriate 
cost^reliability  trade-off. 

The  procedures  contained  in  the  handbook  not  only  optimise  the  screen  selection  and 
placement  but  also  provide  management  with  tools  and  methodology  to  optimise 
resource  allocations  and  to  assess  the  cost  trade-offs  between  defect  prevention 
through  analysis  and  corrective  action,  and  screening. 
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S.  OBTAILBD  GUIOELIMIS 

5.1  ESS  IiiioI«iiigntation  Proceduraa.  Th«  following  paragraphs  outlina  the 
procaduros  raquirad  to  daaign,  iaplamant,  and  monitor  a  factory  ESS  program  with 
tha  objactiva  of  continuously  reducing  dafacts  through  pravantive  actions  such  that 
ESS  can  ba-raducad  to  a  minimum  (and  ideally  eliminated  ascapt  for  that  portion 
raquirad  for  PRVT).  Tha  procedures  are  aimed  at  optimising  tha  combined 
usar/producar  cost  of  achieving  a  raquirad  field  reliability  under  prevailing 
conditions. 

An  ESS  program  consists  of  three  phases  -  planning,  davalopment,  and  production. 

Tha  planning  phase  is  used  to  (a)  design  a  cost  optimised  factory  ESS  program  that 
achieves  tha  raquirad  field  reliability  for  an  existing  dasign  and  defect  density, 
and  to  (b)  create  a  suite  of  quantitative  factory  requirements  that  are 
meaningfully  related  to  the  required  reliability  and  n^e  measurable  and  monitorable 
by  the'  producer.  The  planning  phase  uses  defect  density  and  screening  strength 
data  provided  in  the  handbook  and  industry  and  user's  data.  Since  the  data  are 
approximations,  the  values  must  be  validated  and  refined  during  the  development 
phase. 

During  the  development  phase,  monitor  and  control  procedures  are  used  to 
quantitatively  measure  fallout  data  and  thus  refine  the  estimates  for  defect 
density  and  screening  strength  so  that  appropriate  modifications  can  be  made  for 
production.  Similar  procedures  are  used  throughout  the  production  phase  to  provide 
a  quantitative  assessment  and  fendbac'k  on  whether  or  not  reliability  requirements 
are  being  achieved  and  the  jsxtant  that  continuous  improvement  is  being  realised.. 
If  problems  exist,  the  procedures  assist  in  focusing  on  problem  areas  and/or 
identifying  the  areas  requiring  more  in  depth  or  root  cause  analysis.  The 
methodology  allows  for  a  continual  reduction  in  ESS/screening  as  defect  densities 
are  reduced  through  corrective  action  (provided  customer  reliability  requirements 
•re  satisfied)  and  thus  allows  the  user  and  producer  to  ‘optimize  the  product  cost' 
and  reliability.  The  procedures  also  provide  management  with  the  necessary 
visibility  and  models  for  assessing  the  tradeoffs  between  defect  prevention  and 
screening  and  assessing  the  return  and  effectiveness  of  resource  allocations. 

The  ESS  program  includes  a  PRVT  segment  that  is  used  in  conjunction  with  ESS  data 
analysis  to  provided  the  necessary  confidence  that  field  reliability  will  be 
achieved.  The  PRVT  segment  is  calibrated  during  the  development  phase'  and  is  a 
fixed  segment  of  the  system  level  factory  ESS.  As  defects  are  prevented  through 
corrective  action  and  ESS  consequently  reduced,  the  PRVT  segment  becomes  useful  in 
flagging  possible  but  of  control  conditions  that  could  otherwise  be  missed  (due  to 
the  reduced  ESS).  The  PRVT  segment  thus  serves  multi-purpose  roles  for  ESS 
polishing,  field  reliability  indication,  and  out  of  control  identification. 

It  should  be  noted  that  software' is  available  from  RAOC  which  fully  automates  these 
procedures. 


There  are  a  total  of  six  procedures: 
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a.  Procadura  A  antltlad,  “Optimiaing  Scraan  Salactlon  and  Placamant”  uses 
procaduras  B  and  C  (which  aatimata  dafact  dansity  and  scraaning  strangth 
raspactiwaly )  to  daaign  tha  ESS  program,  and  Procadura  D  to  validata  the 
original  aatimataa  of  dafact  dansity  and  seraaning  atrangth  and  raCina  tha 
program. 

b.  Procadura  B  antitlad,  "Estimating  OaCact  Dansity”  is  usad  to  astimata  tha 
incoming  dafact  dansity. 

c.  Procadura  C  antitlad,  "Estimating  Seraaning  Strangth”  is  usad  to  astimata  the 
seraaning  atrangth. 

d.  Procadura  0  antitlad,  "RaEining  Estimatas  of  Dafact  Density  and  Screening 
Strangth"  is  usad  to  analyza  factory  fallout  data  to  provida  ravisad  astimatas 
of  Din  and  SS. 

a.  Procadura  E  antitlcd,  "Monitor  and  Control*  is  usad  to  provide  a  quantitative 
assessment  of  whether  reliability  requirements  are  being  attained  and  to  what 
antent  continuous  improvement  is  being  raalisad. 

f.  Procadura  P  antitlad  Product  Reliability  Verification  Teat  (PRVT)  is  uped  in 
conjunction  with  Procedure  E  for  Monitor  and  Control  Purposes  to  provida 
confidanca  that  field  reliability  will  be  achiavad.  .  . 

5.2  Procedure  A  -  Qotimiaine  Screen  Selection  and  Placement. 

5.2.1  Objective.  To  plan  an  ESS  program  such  that  the  required  field  reliability 
is  attained  at  an  optimum  combined  user-producer  cost. 

5.2.2  Methodology.  The  field  reliability  is  determined  by  the  latent  defects 
remaining  at  the  time  of  shipment  and  the  asistence  of  non-screenabla  defects  that 
result  in  a  constsnt  failure  rate.  The  objectiva  of  this  procedure  is  to  optimise 
the  cost  of  reducing  the  latent  defect  population  to  an  acceptable  level  defined  as 
that  which  achieves  the  required  field  reliability.  In  planning  an  ESS  program  the 
first  step,  is  to  determine  tha  maximum  allowable  remaining  latent  defects  that 
allow  the  required  reliability  t:3  be  achieved.  Having  determined  the  maximum 
allowable  remaining  defects,  the  required  factory  screening  strength  is  determined 
from  the  estimated  initial  defects  (determined  in  Procedure  B)  by  solving  the 
equation: 

SS  «  Dremoved/DirITIAI. 

Enowing  tha  required  factory  screening  strength,  the  next  step  is  to  optimize  the 
screen  selection  and  placement  based  on  tha  combined  user-producer  cost.  This  is 
accomplished  by  determining  the  cost  of  removing  the  required  number  of  defects 
using  various  ESS  options. 

There  are  essentially  three  stages  for  applying  this  procedure.  During  the  design 
stage  initial  estimates  of  Din  and  S'*  are  derived  using  the  mathematical  modelling 
techniques  of  Appendix  A  and  the  data  included  in  Procedures  B  and  C  augmented  by 
any  prior  production  history  (collected  and  analyzed  according  to  the  techniques  of 
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Pr6c»duc,«  0)  {rom  similar  equipmant.  A  design  safety  margin  is  built  in  to  account 
for  accuracy  limitations  of  the  estimates.  During  the  development  and/or  early 
production  phase,  additional  ESS  may  be  added  to  provide  sufficient  data  to  use  the 
curve  fitting  technique  of  Procedure  D  to  ’calibrate”  the  factors  determined  in  the 
design  stago.  A  minimum  of  two  RV  cycles  and  10  TC  cycles  are  recommended.  During 
the  production  phase  the  curve  fitting  techniques  of  Procedure  D  are  used  to  refine 
and  validate  the  program  on  a  continual  basis. 

5.2.3  Procedure  Steos. 

Procedure  Al.  The  objective  of  this  procedure  is  to  create  the  basic  ESS  model  for 
a  particular  program  and  to  determine  the  incoming  defect  density,  allowable 
outgoing  defect  density,  and  factory  ESS  constraints. 

Step  1.  Determine  the  initial  defects  resident  in  each  assembly  at  all  test  and 
integration  levels  using  the  methods  of  Procedure  B.  Proportion  the  defects  into 
RV  and  TC  sensitive  populations  using  the  ratio  20%  RV,  80%  TC  (ref.  4.10.3.3)  or 
other  suitably  determined  ratio. 

Step  2.  Determine  the  factory  integration  sequence  and  define  all  restrictions  and 
requirements  with  respect  to  assembly,  calibration,  and  acceptance  testing.  For 
example,  determine  exactly  when  in  the '  factory  integration  sequence  any  sub¬ 
assembly  calibratAon  procedure  should  be  performed.  Allow  for  final  ATP  and  test 
over  environment.  Prepare  a  multi-level  ESS  flow  diagram  depicting  the  integration 
and  environmental  testing  requirements  as  illustrated  in  Figure  5.1.  This  diagram 
illustrates  the  production  flow  and  provides  the  framework  for  ESS  selection  and 
placement.  -  — 

Step  3.  Use  Procedure  C  to  determine  the  screening  strength  for  various  ESS 
options  selected  In  Procedure  A3.  The  model  and  calculations  for  a  multilevel  ESS 
flow  are  illustrated  in  Figure  5.1. 

Step  4.  Given  the  customer's  reliability  requirements,  determine  the  maximum 
number  of  defects  permissable  at  the  time  of  shipment  that  allow  this  reliability 
to  be  achieved.  This  is  a  three  stage  process  as  follows; 

a)  Determine  whether  the  equipment  has  a  limiting  MTBF.  The  limiting  MTBF  is 

the  maximum  that  can  be  achieved  and  is  limited  by  the  state  of  the  art 

in  design,  parts,  materials,  and  processes  and  should  be  obtained  from 
experience  data  from  similar  equipments. 

b)  Given  the  customer's  required  MTBF  and  using  a  suitable  design  safety 

margin  to  allow  for  estimation  errors  (typically  l.S  to  2)  calculate  the 

petmissable  failure  rate  (FR)  due  to  latent  defects  using  the  expression 
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OOOT  «  Oil  (X'SSi)  (l-SS3)(l-SSs) 

*  Oi2  (I-SS2)  (l-SS3)(l-SSs) 

>  OI3  {1>SS3)  (l-SSj) 

+  Oi4  (I-SS4)  (1-SSs) 

*  Ois  (l-SSs) 

Vlgurv  S.i  Sampls  Multi  L«v«l  ESS  Flow  Diagram 


FRiduetolateni)  S 


f - ^ 

\  Reauirea 


RequindMTBF  InhenmtMTBF  /  SAFETY  MARGIN 


c)  To  datormina  tha  aaaiaua  nuabac  oC  dafacts  paraisaibla  at  tha  tiaa  of  shipment 
it  is  nacassary  .  to  datormina  tha  ralationship  botwaan  Cailura  rata  and 
remaining  deCeets  OpxjiAiMIin;  as  follows 


©  FR  = 


DField(t) 

t 
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wh«r«  t  ia  tna  paclod  ovar  which  MTBP  is  to  ba  maasurad  and  Opiaid  is  tha  number  of 
field  failures' due  to  latent  defects  occurring  during  the  interval  t. 

®  OfieLO  ■  DremaIMIMG  •  SSfiEtD  (t)  where  SSpxXLO  (t)  is  the  equivalent 

screening  strength  of  the  field  environment 
for  a  period  t. 

®  SSpiEU)  (t)  ■  l-exp(-kt)  where  k  is  the  field  precipitation  rate 

Substituting  (2)  and  (3)  into  (1)  gives 

[1  —  erp( — kt] 

“  ^REMAINING  J 


In  this  expression  DremaimiHG  (the  number  of  dsfacts  remaining  after  factory 
screening)*  t  and  k  are  all  defined  with  respect  to  the  field  stress.  It  is  thus 
necessary  to  modify  this  expression  by  the  stress  adjustment  factor  (SAF) 
determined  in  Procedure  B  to  determine  the  remaining  defects  at  factory  stress.' 


f'*^=DREMAINlNO*SAP 


[1-  exp(-kt| 


This  expression  is  applicable  to  both  the  RV  and  TC  sensitive  defect  populations, 
therefore,  the  complete  expression  is: 


Dg_ (RV)  eSAF  [I  -  exp  ( -  K (RV)  t)|  +  Djjg^AiNtNO  ^ 

FR  - - = - ^ ^ ^ ^ ^ - 

t 

There  ace  two  methods  for  determining  the  value  of  k. 

(i)  If  the  field  application  environmental  stresses  are  known,  calculate  k  using 
the  expressions  given  in  step  1  of  Procedure  C. 

(ii)  If  the  field  application  stresses  are  not  known,  a  suitable  average  value 
(based  on  industry  or  historical  data) '  can  be  used.  Typical  values  for  K  are 
1/500  to  1/2000). 

Step  3.  Include  the  field  model  and  parameters  in  the  test  flow  diagram  created  in 
step  3  to  complete  the  ESS  Model.  Figure  S.2  illustrates  a'  portion  of  a  sample  ESS 
test  flow  diagram. 

Procedure  A2.  The  objective  of  this  procedure  is  to  determine  th«  cost  of  the  ESS 
program,  defined  as  follows: 

ESS  COST  -  FACTORT  TEST  COST  4-  FACTORY  ESS  COST  FACTORY  REWORK  COST  FIELD 
FAILURE  COST. 

Step  1.  Determine  the  cost  of  factory  tasting.  This  should  include  all  equipment 
coats  including  equipment  calibration  and  maintenance,  operation. 
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documantation,  facllitias,  utllitias  (powar,  watar,  liquid  nltroqan  ate.) 
and  labour  costs  includinq  thosa  associatad  with  paeforming  tha  scraan, 
racording,  rasults't  and  parforming  quality  assuranca  and  administrativa 

tasks. 

Datsraina  tha  cost  ot  factory  ESS  at  aach  staga  that  it  is  parfornad. 
Includa  thosa  factors  and  incidantal  coats  nantionad  in  stap  1. 

Oataraina  tha  nuabar  of  dafaets  raaovad  at  aach  staga  of  acraaning  using 
OfUDfOVED  *  DIM  *  SS  (raf  Appandia  A).  This  is  applicabla  to  RV  and  TC 
saparataly.  Tha  RV  and  TC  faults  ara  also  shown  on  tha  taat  flow  diagram 
{raf.  rigura  S.2). 

Stao  4.  Oataraina  tha  avaraga  total  cost  to  rapair  dafaets  at  aach  staga.  This 
cost  will  includa  all  fault  diagnostics,  rawork/rapair ,  ratast,  rapaat 
ESS,  and  data  racording  costs.  Includa  all  tha  incidental  costk 
incurred.  These  should  includa  carrying  costs  for  spares,  additional 
transit  time  for  equipment,  idle  time,  support  and  administration.  Also 
included  in  these  incidantal  costs'  should  be  logistics  support 
eonsidarations  such  as  tha  costs  associatad  with  providing  spares  with  a 
minimum  amount  of  ESS. 

Stao  3.  Oataraina  tha  "dafaet  cost*  by  multiplying  tha  number  of  dafaets  removed 
at  each  stage  by  the  cost  to  rapair  aach  dafact. 

Stao  6.  Consider  tha  user  cost  by  treating  tha  field  as  an  antansion  of  tha  ESS 
test  flow  and  determining  the  user's  cost  due  to  a  defect; 

Stao  7.  Oataraina  tha  total  usar/producar  cost  for  the  ESS  program  as  the  sum  of 
those  coats  determined  in  steps  1  through  6. 

Procedure  A3.  Tha  objective  of  this  procedure  is  to  optimise  tha  combined 
usar/producar  cost  of  achieving  tha  specified  reliability  as  calculated  using 
Procadura  A2.  This  is  accomplishad  by  aalaeting  and  placing  RV  and  TC  screens 
(with  their  raspactiva  strengths  datarminad  according  to  tha  methods  of  Procadura 
C)  at  various  locations  in  tha  ESS  model  created  in  Procadura  A1  and  calculating 
tha  associatad  coat  using  Procedure  A2.  These  costs  and  defects  removed  and 
remaining  can  bn  charted  as  illustrated  in  Figure  S.2. 

Selecting  Assembly  Laval  ESS.  Usually  ESS  at  tha  lowest  level,  i.e., 
part  or  assembly  minimises  cost;  however,  depending  on  detection 
efficiency  and  tha  peculiarities  of  any  particular  electronic  system, 
this  may  not  always  be  tha  case.  Use  Table  4.9  as  a  guide  in  selecting 
tha  initial  assembly  level  ESS. 

Select  systam  level  SS  as  rnquirad  to  achieve  tha  desired  field 
reliability.  Use  Table  4.9  as  a  guide.  Rota  that  at  system  level  RV 
should  always  be  followed  by  TC  to  enhance  tha  detection  efficiency. 


Step  3.  Oatarminn  the  cost  of  this  ESS  plan  using  Procadura  A2. 
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SfD  4.  Idantity  ass«mbli«s  and  modules  with  high  system  level  ESS  cost  and  Cor 
those  specific  assemblies  select  specific  lower  level  (sub-assembly  or 
part  level)  ESS.  Reduce  the  system  level  SS  as  a  result  of  making  these 
changes  ensuring  that  field  reliability  is  achieved.  Recalculate  the  ESS 
cost. 

Step  5.  Repeat  step  4  until  the  program  has  been  optimised  for  cost. 

Procedure  A4  After  optimising  the  screen  selection  and  placement,  it  is  necessary 
to  ensure  that  the  ESS  is  not  having  deleterious  effects  on  the  useful  fatigue 
life.  This  is  determined  by  calculating  the  damage  indes  D  from  the  equation  □  ■ 
RSB  where  N  represents  the  stress  duration,  S  ■  stress  level,  and  B  «  fatigue 
esponent. 

Pot  TCi  N  «  number  of  cycles 

S  >  temperature  range  in  degree  Celsius 
B  a  3.S  (thermal  fatigue  exponent  Cor  solder) 

For  RVl  N  a  duration  of  vibration  (hours  or  minutes) 

S  a  Grms  vibration  level 

B  a  6.4  (vibration  fatigue  exponent  far  solder) 

1  RBF  Crandall  -  Random  vibration,  publisher  -  John  Wiley  and  Sons,  RT 

Engelmaier  -  Effects  of  Power  Cycling  in  LCC;  Bell  Laboratories,  RJ 

Procedure  AS.  Refine  the  program  as  designed  using  A1  through  A4  by  determining 
actual  values  for  Din,  OB,  PE,  and  SAP  from  factory  and  field  data  analyzed  using 
Procedure  0.  The  development  or  early  preproduction  phase  should  be  usdd  to  verify 
and/or  refine  these  original  estimates.  Subsequently,  the  production  data  should 
be  analysed  on  a  regular  basis  to  ensure  the  program  remains  optimum  under  changing 
conditions. 

5.3  Procedure  B  -Estimating  defect  density. 

9.3.1  Objective.  .  Obtain  estimates  of  the  number  of  defects  resident  in  the  system 
prior  to  beginnlne  ESS. 

5.3.3  Methodology.  Various  imperfections  are  introduced  during  the  assembly  and 

integration  of  the  equipment  due  to  state  of  the  art  limitations  in  the  design, 
testing,  and  manufacturing  of  parts  and  assemblies.  The  total  number  of 

imperfections  is  dependent  upon  these  factors;  the  quantity  and  quality 
(technology,  screening,  reliability  level,  etc.)  of  the  parts  used  and  the  assembly 
complexity  (number  of  connections,  processing,  packaging  densities,  etc.)  Unless 
removed  through  factory  ESS  some  fraction  of  these  imperfections  will  precipitate 
under  field  stress  conditions  and  cause  equipment  failure.  The  number  of 

imperfections  that  will  precipitate  is  dependent  upon  the  factors  mentioned  above 
and  the  field  stress  levels.  The  number  of  defects  for  either  factory  ESS  or  field 
must  therefore  be  defined  relative  to  the  applicable  factory  and  field  stress 
levels. 

In  this  procedure  the  number  of  defects  is  defined  relative  to  a  baseline  stress 
level  equivalent  to  RSM  3000  ESS  guidelines.  Appropriate ' factors  are  then  applied 


5-t 


OOO-HDBK-344  (USAF) 


to  dotornino  .tho  nunbor  of  dofects  for  diffarent  stress  levels  of  vibration, 
temperature  and  temperature  transition  rates  th'at  occur  ‘  in  the  factory  and  field. 
It  is  important  to  address  the  stress  adjustment  factors  when  planning  an  ESS 
prograai  since  they  affect  the  economic  optimisation.  Increasing  ESS  stress  levels 
causes  more  defects  to  precipitate  (than  would  normally  occur  in  the  field), 
incurring  added  rework  and  retest  cost.  Reducing  ESS  stress  levels  increases  the 
tisw  required  to  remove  field  defects  thereby  affecting  throughput  and  equipment 
utilisation,  etc.  Note  that  factory  ESS  stress  levels  should  always  be  higher  than 
the  application  stresses. 

To  ensure  that  the  ESS  program  has  an  appropriate  balance  of  RV  and  T/C  stresses, 
the  estimated  defects  must  be  divided  in  groups. 

-  RV  defects  (i.e.,  those  defects  that  can  be  precipitated  by  RV  stress 
only)  , 

-  T/C  defects  (i.e.,  those  defects  that  can  be  p.ecipitated  by  T/C  stress 
only) 

RV-T/C  defects,  (i.e.  those  defects  that  can  be  precipitated  by  .either, 
7/C  or  RV  stress) 

-  Time-Temperature  -  Humidity  -  Bias  (TTHB)  defects,  (i.e..,  those  defects 
precipitated  by  a  combination  of  time,  temperature,  humidity,  and 
electrical  bias) 

-  Mechanical  Shock  (MS),  (i.e.,  those  defects  precipitated  by  mechanical 

shock ) . 

Factory  ESS  is  effective  in  removing  RV,  MS,  and  T/C  sensitive  defects  with  the 
majority  of  TTHB  sensitive  defects  escaping  to  the  field.  Field  defects  thus 

comprise  residual  RV  and  T/C  defects  approaching  a  constant  failure  rate 

distribution  (i.e.,  the  screening  limit).  TTHB  defects  have  a  different  time  to 
failure  distribution  than  T/C  or  RV  defects  and  practical  purposes  are 

considered  non-screenable  and  are  not  addressed  by  this  ESS  program. 

The  procedure  steps  are  as  follows: 

i)  Estimate  defects  for  each  assembly  ahd  the  total  system  at  base  line 

stress. 

ii)  Proportion  the  defects  into  RV  and  T/C  sensitive  populations. 

iii)  Apply  stress  adjustment  factors  to  determine  the  defects  under  different 
factory  stress  levels. 

The  initial  estimates  derived  using  these  procedures  are  only  approximate  and 
should  be  refined  based  on  the  user's  actual  data  obtained  during  the  development 
phase  or  from  the  production  of  similar  equipment.  Adjustment  may  be  required  on  a 
system  or  individual  assembly  basis.  The  procedures  for  refining  the  original 
estimate  are  provided  in  Procedure  D. 

5.3.3  Procedure  steps. 

Procedure  B1  Determine  the  number  of  latent  defects  resident  in  the  equipment  it 
baseline  stress  as  follows: 
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SfB  1.  'Th«  nunbar  of  dofocts  in  a  aystom  dopends  on  tho  quantity  and 

quality  of  parts  and  workmanship  characteristics.  Therefore  it  is 
first  necessary  to  determine  the  system  complexity  which  is 
described  by  a  complexity  matrix.  This  matrix  comprises  the 
individual  complexity  vectors  for  each  assembly  and  sub-assembly, 

including  appropriate  factors  for  multi-use  assemblies.  These 

complexity  vectors  are  defined  by  the  quantity  of  parts  used  in 
various  part-type  reliability  categories  and  the  quantity  of 
manufacturing  (assembly  and  soldering)  characteristics  present  as 
defined  in  MIL-STO-20Q0.  A  sample  assembly  complexity  vector  is 
shown  in  Figure  9.3. 

Step  2.  Determine  the  initial  number  of  defects  at  baseline  stress  by 

multiplying  the  system  complexity  matrix  by  the  baseline  stress 
defect  density  vector.  The  baseline  stress  defect  density  vector  is 
determined  from  Table  5.1  or  from  prior  industry  and  user  data.  A 
sample  system  defect  density  breakdown  is  shown  in  Figure  5.4. 

Step  3.  Proportion  the  total  defects  into  populations  that  are  sensitive  to 
RV  and  TC.  This  improves  tho  modelling  accuracy  and  ensures  a 
suitable  balance  of  RV  and  TC  is  achieved.  Studies  indicate  that 
typically  204  of  the  total  defects  are  sensitive  to  RV  and  80%  to  TC 
(reference  4.10.3.3). 


ROTE;  The  population  of  RV  and  TC  defects  is  based  on  the  total 
population  and  not  the  factory  fallout.  Some  factory  fallout  is 
affected  by  the  relative  RV  and  TC  screening  strengths. 
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Tabla  5.1  Baaalin*  Stress  Dsfset  Osnsity  Vectors  (ppm) 
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Procaduc*  B2  Oataraina  tha  strass  adjuatuant  factor  calating  dafects  at  factory 
(baaalina  strasa)  lavala  to  dafacta  at  tha  fiald  application  atreaa  levala  aa 
folloiia: 

SfP  1. 


Stao  2. 


Stao  3. 


9.4  Procadura  C  -  Eatimatinq  Screanino  Stranoth 

9.4.1  Obiactiva.  Xatiawta  tha  nunbar  of  flawa  pracipitatad  and  datactad  (ramovad) 
by  BBS. 

9.4.2  ilathodoloqy.  Tha  acraaninq  atranqth  ia  charactarizad  by  a  pracipitation 
tarn  ^nd  a  dataction  tara  and  datarminaa  tha  fraction  of  aaiating  flawa  that  aca 
raaovad  by  SSS.  Tha  pracipitation  and  dataction  taraa  ara  aatiaated  separately  and 
it  is  their  product  that  datarminaa  tha  screening  strength. 

Precipitation  ia  defined  as  tha  conversion  of  flaas  with  seme  residual  strength 
(latent  defect)  into  a  flaw  with  no  strength  (patent  defect)-  for  example  the 
propagation  of  a  crack  through  a  wire  until  the  wire  is  broken.  The  application  of 
atrass  pracipitataa  a  certain  fraction  of  tha  existing  flaws.  This  fraction  is 

aaaaawd  to  be  constant  for  a  specified  stress  level  and  duration  and  tha 
sHitw— sties  are  discussed  in  more  detail  in  Appendix  A.  A  previous  study  ref. 
BAOC-TB-SB-149  has  determined  tha  pracipitation  effectiveness  of  various  stress 
typM  asBi  has  developed  mathematical  expressions  for  each.  These  expressions  and 
tepreaesaastive  tables  ara  provided  in  Tables  5.14  to  5.17.  As  in  the  estimation  of 
dm^ects,  the  original  estiauita  based  on  these  tables  is  only  approximate 
and  macs::  <ae  validated  or  refined  based  on  actual  user's  data. 


Usa  tables  5.2  through  5.13  to  datarmina  tha  defect  density  vector 
at  tha  anticipated  field  stress  level.  These '  tables  represent  tha 
defect  density  for  different  application  anvironmants  and  were 
derived  from  fiald  data.  For  factory  ESS  planning  purposes,  these 
tables  should  ba  rescaled  to  also  include  the  defects  removed  by 
factory  screening.  A  factor  of  1.5  is  typical.  Multiply  tha  system 
complexity  matrix  from  Procedure  B1  stpp  1  by  the  field  .  strass 
dafact  density  vector  to  datarmina  the  initial  number  of  defects,  at 
the  anticipated  field  stress  levels. 

Oetaraina  tha  fiald  stress  adjustment  factor  as  tha  ratio  of  the 
number  of  defects  at  the  field  stress  level  to  the  number  of  defects 
at  the  baseline  stress  level  (from  Procedure  dl). 

If  factory  stress  levels  do  not  conform  to  the  base  line  stress 
levels  (defined  as  6  Grms  and  4  dogrees  C/min)  apply  a  suitable 
factory  stress  adjustment  factor  (SAP)  ar.  follows: 


For  RV  SAF  =  ( 


actual  Grma 
BGrms 


n  typically  0.5  to  1 .0 


_  actual  transition  rate  .  .  .. 

ForTC  SAF's  (  ■■  —  --  ■  -  )"  n  typically  0  to  .5 

4aegreesC/min 
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TBbl*  S.2  Nicrocisctranie  0«vic«>  0«fact  Oanaity  (In  PPM)  foe  various 

Bnvlronmonts 


OitsMty 
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Tabl«  5.3  Tr*n»l«tor  D€vic««  0«e«ct  Density  (in  PPM)  foe  various  Environmanti 
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Tabla  S.S  Rastscor  Oavicai  Dafact  Oansity  (in  PPM)  for  various  Bn/ironnants 
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Table  S.6  Capacitor  Defect  Oenaity  (in  PPM)  for  various  Invironnents 
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Tabl*  5.7  Magnetic  0«{8ct  Danaity  {in  PPM)  for  various  Invironmants 
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Frieipitation  -by  Itsalf  docs  not  ansura  that  Claws  can  be  detected.  In  many 
instances  a  concurrent  stress  may  be  required  to  detect  and  isolate  the  failure. 
For  esample,  a  broken  wire  may  make  intermittent  contact  at  low,  ambient  stresses. 
Also,  depending  upon  the  function  affected,  Che  defect  may  only  cause  degraded 
performance.  Either  condition  could  require  dxtended  testing  and  may  require 
concurrent  stress.  The  capability  of  detecting  a  patent  defect  is  measured  by  the 
detection  efficiency.  ^ 

The  removal  of  a  potential  defect  or  Claw  requires  the  Claw  to  be  precipitated  and 
subsequently  detected  and  removed.  The  detection  efficiency  is  defined  as  the 
capability  of  detecting,  isolating  and  removing  the  i  defect  once  it  has 
precipitated.  It  is  a  measure  of  the  extent  that  factory  testing  exercises  all 
possible  field  applications  and  conditions  and  is  the  product  of  the  following 
factors  and  considerations: 

(a)  Probability  of  observing  functional  ana  parametric  d'iCects  (i.e., 
probability  of  detection  x  probability  of  occurrence) 

(b)  Necessity  for  concurrent  stress 

(c)  Probability  of  isolating  and  then  removing  the  defect  without  creating  an 
additional  defect(s) 

Studies  indicate  that  a  large  fraction  of  defects  require  concurrent  stress  to  be 
detectable.  Therefore  ESS  that  does  not  employ  testing  during  stress  application 
is  relatively  ineffective.  It  is  also  the  reason  that  RV  sfress  should  be  followed 
by  T/C.  RV  is  relatively  short  in  duration,  thus  the  detection  efficiency,  which 
has  a  Poisson  (1  -  distribution  may  be  inadequate. 

5.4.3  Procedure  Steps 


Step  1.  Determine  the  precipitation  efficiency.  Expressed  as  a  function  of 
stress  duration,  the  precipitation  efficiency  is  given  by  1-exp  (-kt)  where  t 
is  the  stress  duration  in  hours,  cycles,  etc  and  k  is  a  stress  constant 


0.0017  (^T  •••  .6)>6  [ln(RATE  *  2.718)]?  where 

Tmax  ~  l^min  in  degrees  C  and  RATE  >  degrees  C/minute 


determined  for  each  type  < 

Temperature  Cycling 

k  -  ( 

AT  ■  ' 

Constant  Temperature 

k  >  ( 

Random  vibration 

k  «  ( 

Swept  Sine  Vibration 

k  -  ( 

Fixed  Sine  Vibration 

k  -  ( 

Tables  5.14  to  5.17 

provide 

6  in  Grms 


0.000727GO*863 
0.00047  g0*<9 


screening  parameters.  For  RV  screens  it  is  necessary  to  include  an  axis 
sensitivity  factor.  RV  applied  in  the  axis  perpendicular  to  the  plane  of  the  board 
will  have  tne  greatest  effect.  When  selecting  and  modelling  RV  stress,  the 
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precipitation  oCeicioncy  is  thus  ^ivon  by  (1  -  sxp  (-kt)l*  AXIS  SENSITIVITY  FACTOR 
whsrs  the  axis'  sensitivity  factor  is  the  defect  density  component  in  the  sensitive 
axis  divided  by  the  total  defect  density.  Transmisibility  and  resonance  effects 
must  be  considered  and  the  frequency  spectrum  may  need  to  be  suitably  notched  to 
avert  overstress  or  wear-out  effects.  Similarly  thermal  mass  and  conductivities 
must  be  considered  when  determining  TC  transition  rates  and  required  dwell  times. 
The  stress  levels  for  all  these  equations  pertain  to  the  equipment  being  screened 
and  not  the  chambers  etc. 


It  should  also  be  noted  that  the  expressions  and  tables  for  precipitation 
efficiency  are  only  approximate  and,  as  in  the  estimation  of  initial  defects, 
should  be  refined  based  upon  actual  users  data  according  to  the  techniques  of 
Procedure  0.  . 

Step  2.  Determine  the  detection  efficiency  (OE)  The  OE  term  is  sensitive  to  three 
factors  and  must  be  estimated  accordingly.  These  three  factors  .(and  their 
respective  range  of  values)  are: 


a)  Type  of  testing  performed: 

Functional  only 
Functional  and  parametric 

b)  Environmental  conditions  during  tost: 

Testing  performed  under  ambient  conditions  only 
Testing  performed  concurrently  with  stress 

e)  The  ability  to  observe  and'  isolate  the  defect 
and  the  probability  of  successfully  removing 
the  defect  without  introducing  another 


O.S  to  0.8 
0.8  to  1.0 


0.2  to  0.6 

1.0 


0.8  to  1.0 


The  product  of  these  factors  is  the  detection  efficiency. 

Step  3.  Determine  the  screening  strength  as  the  product  of  the  precipitation 
efficiency  and  detection  efficiency. 


EXAMPLE  OF  SS  CALCDLATIOW 

For  TC  of  4  cycles  at  S*C/minute  over  a  lOO’C  range,  PE  »  .6027  (from  Table  5.15) 


For  RV  of  S'minutes  at  5Grms,  PE  «  .303  (from  Table  5.16) 

.9 

1 

.95 

The  detection  efficiency  is  0.9  x  1.0  x  .99  «  .855 

SS  (TC)  «  PE  X  DE  »  ;6027  •  .855  »  0.5153 
SS  (RV)  -  PE  X  DE  *  .303  •  .855  ■  0.259 


Given  the  following  "DE*  factors  functional  and  parametric  Test 

Test  during  environmental  stress 
Probability  of  detecting,  isolating 
and  reamving  the  defect 
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Tabl*  S.'12.  PWB  D«£act  Dansity  (in  PPM)  for  varioua  Knviroitmants 


Quality  L(«t1 


C««1ronatRt 


«It-SPtC 


Laaar 


fiQ 

425.0 

490.3 

CM 

1792.4 

HP 

1429.2 

NSI 

10'.:.  7 

NS 

1302.4 

m 

2470.0 

m 

2174. 1 

mitt 

3071.2 

Attt 

4091.7 

Ate 

731.1 

1139.3 

Alt 

I  1153. 7 

AIA 

!  1547.9 

Alf 

2241.0 

AUC 

1751.4 

AUT 

3202.3 

AM 

5323.3 

AUA 

4302.0 

AM 

7344.2 

S5 

425.0 

mr 

1994.5 

NPA 

2470.0 

BSl 

9527.3 

ML 

4139.4 

a 

102247.9 

42S0.0 

I403.2 
n«4.3 
U291.S 
10574. « 
13024.0 

24700.3 

21741.2 

30702. 2 

40914.5 
7311.4 

11393.2 

11534.5 

15479.2 

22411.4 

17514.1 

32123.1 

53232.5 

43027.1 
73441.9 

4250.0 

19945.2 

24700.3 

55273.5 

41394.3 
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Tabla  S.13.  Manufacturing  Characteristics  (in  ppm)  in  Various  environments 


eMVIROMMEMT 


MIJlUFACTURIMG  CHARACTERISTICS 


ASSEMBLY 


0.9 
0.9 
4.9 

4.9 

1.9 

2.9 
6.8 
6.86 
7.84 
10.78 
1.96 
2.94 
3.92 
2.94 
4.90 
1.96 
3.92 
4;90 
4.90 
3.88 
0.98 
4.90 
6.36 

14.71 

16.67 

274.91 


1.25 

1.25 

6.27 

6.27 

2.51 

3.76 
8.78 
.  8.78 
10.03 
13.79 

2.51 

3.76 
5.01 

3.76 

6.27 

2.51 
5.01 

6.27 

6.27 

7.52 

1.25 

6.27 
8.78 
18.81 
21.31 
351.04 


AccaUratlon  U««t  (G-RNS) 
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Table  S.IS.  Precipitation  ICfieiencT  Pactoca  -  Teepecature  Cycling  Screens 


Tsassrstort  tsngs  4  (^C) 

SuMtr 

mm 

9t 

100. 

Cyelts 

sssi 

20. 

40. 

40. 

40. 

120. 

140. 

140. 

180. 

2 

e 

T 

5 

.1433 

.2345 

.2444 

.3324 

.3457 

.4023 

.4312 

.4572 

M 

10 

■.2501 

.4031 

.4412 

.5410 

.5451 

.4250 

.4425 

.4520 

.7173 

IS 

.3111 

.5254 

.4.24 

.4752 

.7232 

.74)2 

.7520 

.4175 

20 

.4702 

.4155 

.7034 

.7434 

.4075 

.4407 

.4445 

.1471 

.5037 

4 

e 

T 

S 

.2111 

.4147 

.4535 

.5543 

.4027 

.4427 

.4745 

.7054 

.7305 

10 

.4444 

.4437 

.7304 

.7443 

.8312 

.1424 

.4443 

.5051 

.5201 

15 

.42S2 

.7744 

.4454 

.4545 

.5234 

.5430 

.5547 

.5447 

.5740  1 

A 

20 

.7144 

.4522 

.5120 

1 

.5441 

.5425 

.5744 

.5422 

.5473 

.5507  1 

] 

% 

e 

T 

5 

.4141 

.5522 

1 

.4400 

.7025 

.7454 

.7444 

.4140 

.4401 

'  i 

• 

.8401 

10 

.4431 

.7473 

.4403 

.5033 

.5304 

.5445 

.5417 

.5704 

.5774 

IS 

.774? 

.4431 

.5414 

.5457 

.5744 

.5444 

.5510 

.5535 

.5554 

A 

20 

.8S17 

.5432 

.4735 

.5448 

.5525 

.5540 

.5574 

.5584 

.5551 

S 

e 

T 

S 

.S044 

1 

.4574  •  .7435 

.4014 

.1422 

.4723 

.m3 

.5132 

.5274 

10 

.744* 

.4731 

.5275 

.5554 

;571S 

.5411 

.5471 

.5510 

.5534 

IS 

.1425 

.5443 

.5774 

.5845 

.5541 

.5547 

.5541 

.5585 

.5553 

20 

.52)5 

.5741 

.5423 

.5545 

.5544 

.5554 

.55J7 

.5558 

.5555 

10 

e 

T 

.1 

5 

.5414' 

.7375 

.4174 

.H74 

.5005 

.5237 

.5405 

.5525 

.5423 

10 

.4204 

.5242 

.5424 

.5754 

.5483 

.5530 

.5554 

.5572 

.5582  ' 

15 

.4143 

.4754 

.4413 

.5544 

.5544 

.5552 

.5554 

.5558 

.5555  i 

20 

.5545 

.5414 

.5477 

.5553 

.5557 

.5555 

.5555 

.5555 

.5555  ' 

u 

e 

T 

5 

.4544 

.7444 

.4704 

.5115 

.5373 

.5544 

.5441 

.5744 

.5804 

10 

.8724 

.5544 

.5405 

.5504 

.5552 

.5574 

.5545 

.5551 

.5555 

IS 

.5440 

.5444 

.5544 

.H48 

.5554 

.5554 

.5555 

.5555 

.5555 

20 

.5740 

.55U 

.5553 

.5554 

.5555 

.5555 

.5555 

.5555 

.5555 
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5.5  ProcBduf  D  -  Ratining  Eatiaates  ot  D*f*ct  Penalty  and  Screaning  Strength 

5.5.1  Obi«etivg.  To  r«£in«  the  estiaates  of  ESS  modelling  pacametecs  (Oxn>  SS) 
using  actual  factory  and  field  data. 

5.5.2  Methodology.  The  most  important  pareuneter  for  ESS  is  the  defects  remaining 
at  the  time  of  shipment  since  this  determines  the  field  reliability.  Other 
significant  parameters  are  the  initial  defect  density,  and  the  screening  strength 
of  various  screens.  The  difficulty  however,  is  that  none  of  these  parameters  are 
directly  observable  by  the  producer.  Only  the  defects  removed  through  factory  ESS 
can  be  measured.  This  procedure  provides  the  means  for  determining  these  other 
critical  parameters  from  the  factory  data. 

5.5.3  Procedure  Steps. 

Step  1.  Collect  the  necessary  factory  (test  and  failure)  data  from  a  FRACAS 
system.  When  assembling  this  data  it  is  imperative  to  distinguish  between  errors 
and  defects.  Errors  are  preventable  and  usually  detectable  without  environmental 
stress.  The  primary  concern  of  this  procedure  is  the  elimination  of  latent 
defects.  The  minimum  data  requirements  for  the  FRACAS  system  are: 

ESS  Test  Equipment 

Stress  Duration  -  ETI  at  start  of  teat,  completion  of  test,  and  time  of  failure 
Teat  Type 

■umber  of  units  tested  >  number  that  pass/fail 
Assembly  -  Sub  Assembly  >  part  failed 
Failure  Cause 

Step  2  To  determine  Dim,  PB,  and  Ouzmaiming*  curve  fit  the  factory  fallout  data 
(determined  in  step  1)  and  field  data  to  the  following  expression 

Ofallout  •  DE  (DpAT  ♦  (l-exp(-kt)]  CFR.t] 

The  derivation  of  this  expression  is  discussed  in  Appendix  A  and  RAOC  technical 
report  TBO  and  has  been  found  to  be  adequately  representative  of  the  real  world. 

Collect  the  fallout  data  for  each  type  of  environment  (i.e.,  temperature  cycling, 
random  vibration  etc.)  separately  and  prepare  graphs  with  the  cumulative  . defect.*:, 
normalized  as  defects  per  system  as  the  ordinate,  and  the  stress  duration  as  the 
abscissa.  Several  methods  are  available  to  curvefit  this  data  (to  determine  the 
detectable  precipitated  defects)  and  PC.  software  is  available  from  RADC  for  this 
purpose. 

Recalling  Procedure  C,  screening  .  strength  is  the  product  of  precipitation 
efficiency  and  detection  efficiency.  Curvefitting  the  factory  fallout  data  to  the 
above  expression  yields  the  "k"  value  for  the  precipitation  efficiency  expression 
[l-exp(-kt)]  and  also  determines  DPAT  and  CFR.  If  field  data  is  available,  the 
Stress  Adjustment  Factor  (SAF)  and  the  detection  efficiency  of  factory  E.*!S  can  also 
be  determined.  The  SAF  can  be  calculated  by  dividing  the  latent  defects 
(determined  by  curvefitting  the  field  data),  by  the  escaping  latent  defects 
(determined  by  curvefitting  the  factory  data).  The  factory  detection  efficiency  can 
be  calculated  by  dividing  the  field  patent  defects  (determined  by  curvefitting  the 
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Ciald  data)  by  tha  total  factory  fallout.  Thaaa  calculations  can  bo  oxpcessod 
■athoMtically.aa  follows: 

SAT  >  Dlat  (field)  /  (Olat  ( factory) . (1-SS( factory) )] 

01  (factory)  ■  Opat  (field)/  [Factory  fallout] 

Figure  S.S  illustrates  a  sample  curvefitting  analysis. 


5.6  Procedure  1  -  Monitor  and  Control 

5.6.1  Objective.  To  implement  a  program  to  monitor  and  control  the  KSS  program 
(consistent  with  TQM  philosophy)  thereby  ensuring  that  the  program  remains  cost 
effective  under  the  evolving  conditions. 

5.6.2  Methodology.  The  parameters  of  interest  for  monitor  and  control  are  DIN, 
SS,  and  OpzmaiiiihG  snd  srs  determined  in  Procedure  S.  Modified  SPC  and  PARETO 
charts  are  prepared  to  monitor  these  parameters  against  the  requirements 
established  in  Procedure  A. 

5.6.3  Procedure  ^teos. 

Step  1.  Management  monitor  and  control  is  accoaplished  by  preparing  SPC  charts  for 
the  important  parameters  determined  in  Procedure  0.  Since  the  objective  of  CSS  is 
continuous  improvement  in  the  elimination  of  defects  and  their  causes,  a 
homogeneity  test  is  not  adequate  for  Dm  and  DpsiiAIRING  since  they  should  be 
decreasing  with  time  and  product  maturity.  The  SPC  charts  must  also  reflect  the 
requirements,  (which  are  directly  related  to  field  reliability  by  Orxmaimimc)  the 
current  level  of  performance,  and  the  statistically  expected  variation  due  to 
limited  sample  sise.  Mith  conventional  SPC  charts  the  parameter  of  interest 
remains  relatively  constant,  so  that  the.  process  average  (p)  variation  can  be 
determined  by  taking  the  mean  over  many  samples.  For  ESS  however,  the  parameter  of 
interest  (p)  is  expected  to  be  improving  with  time  making  it  necessary  to  use 
regression  analysis.  A  second  order  polynomial  regression  analysis  is  usually 
adequate. 

Figure  5.6  illustrates  a  modified  SPC  chart  for  monitoring  the  total  incoming 
defects  Dxh.  It  displays  the  TQM  goals  (inherently  established  as  part  of 
Procedure  A)  and  compareir  actual  results  to  these  requirements.  The  expected 
statistical  variation  due  to  the  limited  sample  sise  is  calculated  using  a  Poisson 
distribution  and  has  a  standard  deviation  given  by  5  =  Vp/n  where  n  is  the  sample 
sise  and  pis  determined  from  the  regression  analysis. 

Step  2  As  a  supplement  to  the  SPC  charts  created  in  step  1  it  is  sometimes  useful 
to  generate  a  PARETO  chart  to  display  a  breakdown  cf  failure  causes.  The  PARETO 
typically  examines  the  frequency  of  various  causes  of  non-conformities  and 
indicates  defect  frequency  and/or  frequency  percentage.  The  PARETO  identifies  the 
most  frequent  cause  but  not  necessarily  the  most  isiportant  cause  and  can  over  look 
what  is  expected  based  upon  other  considerations,  for  sxamplo  complexity. 

To  overcome  this,  a  modified  PARETO  is  recommended  that  charts  not  only  actual 
results,  but  compares  them  with  the  expected  results  based  on  complexity  and 
statistical  significance.  When  reviewing  the  ?areto  diagram,  situations  where 
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daCtcts  arc  aithcr  greater  or  less  than  expected  require  more  in  depth  analysis. 
Since  a  design  fault  is  often  specific  to  a  particular  assembly,  the  PARETO  chart 
can  help  identify  potential  design  flaws  by  identifying  assemblies  with  defect 
densities  above  expected  levels.  Conversely,  assemblies  with  defect  densities 
significantly  lower  than  expected  levels  may  indicate  low  screening  stresses  or  low 
detection  efficiency.  If  the  SPC  chart  for  a  particular  assembly  continually 
trends  above  or  below  expected  levels  a>id  detailed  failure  analyses  do  not  reveal 
abnormal  causes,  then  the  cause  may  be  to  do  packaging  density  etc.  If  so,  the 
expected  goals  and  requirements  for  the  particular  assembly  can  be  adjusted  by  a 
suitable  correction  factor  and  Procedure  A  repeated.  Figure  5.7  illustrates  a 
typical  application  of  a  PARETO  chart.  To  allow  for  statistical  variations  due  to 
sample  sise,  the  expected  values  are  indicated  as  ±3  sigma  bars  (assuming  a 
Poisson  distribution  as  for  the  SPC  charts).  This  makes  it  possible  to  identify 
not  only  assemblies  with  high  and  low  defect  densities  but  also  those  assemblies 
where  the  defect  density  is  significantly  different  than  expected. 

The  PARETO  is  recommended  because  it  not  only  charts  actual  results,  but  compares 
them  with  expected  results  based  on  complexity  and  statistical  significance. 


5.7  Procedure  P  -  Product  Reliability  Verification  Test  (PRVT) 

5.7.1,  Objective.  To  retain  a  minimum  ESS  so  that  field  reliability  can  be 
projected  and  out  of  control  conditions  identified. 

5.7.2  Methodology.  The  Monitor  and  Control  Procedures  of  Procedure  0  determine 
whether  or  not  outgoing  reliability  requirements  are  being  met  by  comparing,  actual 
factory  results  with  the  goals  (established  in  the  design  stage  of  Procedure  A)  via 
SPC  and  PARETO  charts.  Since  these  procedures  ensure  reliability  is  achieved,  any 
further  testing  would  bO  redundant.  Recall  that  the  ESS  program  operates  in  a 
feedback  loop.  The  intent  is  to  precipitate  defects  (in  the  factory)  that  would 
have  occurred  in  the  field  and  thus  identify  their  causes  so  that  corrective 
aetioi a  can  be  taken  to  prevent  their  recurring.  This  should  continually  reduce 
defect  density  thereby  allowing  for  a  reduction  in  ESS.  However,  the  extension  of 
this  process  is  to  completely  eliminate  ESS  -  thereby  creating  a  situation  in  which 
, there  is  no  mechanism  to  indicate  when  the  process  is  not  in  control  and 
reliability  is  not  being  achieved.  PRVT  is  defined  as  that  portion  of  ESS  retained 
for  the  purpose  of  providing  such  a  mechanism  and  is  inherently  part  of  the  ESS 
program  (and  subject  to  the  Monitor  and  Control  Procedure  of  Procedure  0). 

Any  assessment  of  reliability  must  be  made  on  the  basis  of  the  performance  of  the 
collective  population  in  the  field  and  the  percentage  of  systems  that  are  defective 
in  a  specified  operating  period  (the  assessment  of  factory  performance  must  be  made 
on  the  same  basis).  It  is  measured  by  implementing  a  monitor  and  control  program 
based  on  normalised  parameters  (defects/system,  defects/unit  etc).  The  PRVT 
segment  is  to  be  monitored  in  this  way.  However,  first  pass  yield  (where  first 
pass  yield  is  defined  as  the  number  of  systems  completing  the  PRVT  segment  with  no 
failures  divided  by  the  total  number  of  systems  first  time  submitted)  is  also 
applicable.  Provided  the  defects  are  Poisson  distributed,  first  pass  yield  and 
defects/system  requirements  are  related  by  Yield  >  exp  (-Defects). 
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Figure  5.6  Saaple  SPC  Chart 
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It  th«  SPC  r«(juir«m*nts  (sea  Procedure  E  paragraph  5.6)  and  Che  PRVT  requirements 
of  first  pass  yield  are  not  achieved,  the  outgoing  system  defect  density  is  too 
high  and  corrective  action  must  be  taken  that  addresses  the  general  population.  If 
defect  causes  can  not  be  immediately  removed  to  attain  an  "in  control”  situation, 
ESS  must  be  increased  for  all  production  (subject  to  the  damage  restrictions 
butlined  in-Procedure  A)  until  "control"  is  re-established. 


5.7.3  Procedure  Steps. 


Step  1.  Using  the  mathematical  derivations  relating  first  pass  yield  to  field 
reliability  (MTBP)  detailed  in  Appendix  B,  determine  if  the  first  pass  yield  is 
worse  Chan  required.  If  the  first  pass  yield  and  the  monitor  and  control  technique 
of  Procedure  E  indicates  Chat  Che  necessary  field  reliability  is  not  being 
achieved,  add  ESS  according  to  Che  methods  outlined  in  Procedure  A. 


Step  2.  As  the  ESS  program  evolves  and  ESS  is  reduced,  ensure  that  as  a  minimum, 
one  RV  and  two  TC  cycles  are  retained  for  the  PRVT  segment  to  help  identify  out  of 
control  conditions  that  would  otherwise  be  missed. 
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APPENDIX  A 

Stress  Screening  Mathairatieal  Model 

10.  General.  The  fundamental  objective  of  a  stress  screening  program  is  to  reduce 
the  number  of  latent  defects  in  a  production  lot  of  equipment  to  an  acceptable 
level  by  use  of  cost  effective  screening  regimens.  As  basic  principles,  one  would 
like  to  be  able  to  use  strong  screens  and  efficient  tests,  within  prescribed  cost 
constraints,  which  have  a  high  probability  of  precipitating  and  detecting  defects 
and  thus  achieving  reliability  objectives.  To  transform  these  principles  into 
quantitative  procedures,  it  is  necessary  to  define  various  measures  and  their 
relationships  to  the  screening  process.  This  Appendix  defines  a  mathematical  model 
that  predicts/establishes  relationships  between  quantities  such  as  defect  density, 
precipitation  efficiency,  detection  efficiency,  screening  strength,  and  yield. 

20.  Reference  documents.  (See  Section  2) 

30.  Definitions  and  acronyms.  (See  Section  3) 

40.  General  mathematical  relations. 

40,1  Defect  density.  Under  reasonable  assumptions  that  the  number  of  latent 
defects  in  a  product  are  independently  and  identically  distributed,  the  number  of 
defectives  in  an  equipment  can  be  described  by  the  Binomial  Probability 
distribution,  with  parameters  N  and  P. 

Where  N  «  total  number  of  parts  in  the  equipment 

P  ■  average  part  fraction  defective  over  all  part  types 

A  part  as  defined  herein,  is  any  identifiable  item  within  the  product  which  can  be 
removed  or  repaired,  (e.g.,  discrete  semiconductor,  resistor,  integrated  circuit, 
solder  joint,  connector).  For  large  N  and  small  P  the  Binomial  can  be  approximated 
by  the  Poisson  distribution  with  the  parameter  0  *  BP 

Where  D  •  Defect  Density  (average  number  of  latent  defects  per  item) 

The  defect  density  0  *  RP  can  also  be  represented  as: 

__  m  (A-l) 

D  =  .\P  =  y  n.  p. 

where:  ni  «  quantity  of  each  part  type  i 

Pi  >  defect  density  for  each  part  type  i 

m  a  number  of  different  part  types 

The  procedures  contained  in  Procedure  B  of  Section  5,  for  obtaining  planning 
estimates  of  defect  density,  are  based  upon  the  mathematical  relations  just 
described. 
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40.2  Pfcioitation  ECficienev.  Th«  Precipitation  SCficiency  (PE)  of  a  screen  is 
expressed  as  the  probability  that  the  screen  will  precipitate  a  defect  to  a 
detectable  state  9iven  that  a  defect  susceptible  to  the  screen  is  present.  For  ESS 
to  be  viable,  the  screening  strength  (and  hence  precipitation  efficiency)  of  a 
screen  must  be  independent  of  the  number  of  defects  and  when  the  screen  is 
performed.  -  Mathematically  this  can  be  satisfied  if  the  defects  are'  exponentially 
distributed  in  time. 

0«  -  Olat  <*>  U  -  exp(-kit)l  (A-2) 

where:  Ox  •  defects  precipitated 

OLAT  -  latent  defect  population  at  time  T 
kx  >  stress  constant  for  precipitation 

t  >  stress  duration 

Precipitation  efficiencies  for  various  screen  parameters  are  given  in  Tables  5.14 
through  5.17. 

40.3  Detection  Efficiency.  In  general.  Detection  Efficiency  can  be  represented  by 
a  Poisson  Distribution  in  stress  duration. 

DB(t)  -  OB  X  OPAT  [1  -  axp(>k2t))  (A-3) 

shere:  OB  ■  detection  effici«ncy 

OpAT  ■  existing  patent  defects 

kg  >  stress  constant  for  detection  efficiency 

t  *  stress  duration 

Provided  the  k  terms  for  precipitation  efficiency  and  detection  efficiency  are 
significantly  different,  the  lower  k  term  will  dominate.  Mith  this  simplification, 
detection  efficiency  can  be  considered  to  be  independent  of  t  and  represented  by 
the  constant  OE.  Fixed  parameters  for  calculating  detection  efficiencies ' are  given 
in  Procedure  C  in  Part  5. 

40.4  Screening  Strength.  The  screening  strength  (SS)  is  defined  as  the  product  of 
precipitation  efficiency  and  detection  efficiency. 

SS  >  PE  X  OE  (A-4) 

40.5  field.  Given  prior  estimates  of  pi,  equation  A-1  can  be  used  to  estimate 
OlR>  fhe  incoming  latent  defect  density  before  assembly  screening,  since  M  and  ni 
are  known  for  the  assemblies  and  equipment  to  be  screened.  The  remaining  defect 
density  D^xjiaiiiimg  o**  described  in  a  similar  manner,  except  that  the  pi,  of 
equation  1,  would  be  interpreted  as  the  remaining  part  defect  density.  0i]|  and 
OremaiMING  normalised  quantities  and  can  also  be  expressed  as: 
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total  number  of  latent  defects  intnxiuced 
total  numberof equipment  in  the  Lot 


_  total  number  of  latent  defects  remaining 
REMAINING  total  number  of  equipment  in  the  Lot 

Without  an  ESS  progran,  a  production  lot  of  oquipaonts  uxll  contain  defects  which 
are  introduced  into  the  equipments  as  escapes  from  previous  part  level  screens  and 
by  poor  workmanship  or  manufacturing  processes.  The  defects  introduced  are 
expressed  quantitatively  as  the  average  number  of  defects  per  equipment  (Dm  or 
defect  density).  Using  the  Poisson  probability  distribution,  thu  probability  that 
an  equipment  is  defective  P(0)  (i.e.,  contains  one  or  more  defects)  is  given  by: 

P(D)  «  1  -  e““lW  (A-5) 

The  objective  of  an  ESS  program  is  to  reduce  Dm  to  an  acceptable  level,  say 
OreMAIMING*  wK?re  OremaintnG  defined  as  the  average  number  of  defects  remaining 
per.  equipment  at  delivery  to  the  customer.  Reducing  Dim  to  Dremaining  also  reduces 
P(D)  so  that: 


P(D)  ■  1  -  «“®REMAI1IIHG  assuming 

all  remaining  defects  will  fail  (A-6) 

The  probability  that  an  equipment  will  pass  a  screening  test  is  called  Yield. 
Because  not  all  reisaining  defects  fall  during  the  screen,  the  expression  for  yield 
becomes: 

Yield  -  «-0REM0VED  (A-7) 


If  the  Yield  is  specified  as  a  goal,  then  Dremqved  on  t>e  determined  by: 

Drxnovxo  ■  “in  (Yield)  (A-8) 

ami  used  as  an  objective  for  which  an  ESS  program  can  be  planned,  implemented  and 
subsequently  monitored  and  controlled.  Both  DremainimG  nnd  Yield  are  used  in  the 
handbook  Procedures  A  and  P,  as  the  quantitative  goal  of  the  ESS  program. 

40.6  Remaininq/Ramoved  Defects.  The  quality  at  the  ESS  program  and  by  extension, 
the  number  of  defects  removed  is  a  function  at  five  simultaneous  effects:  . 

-  pfecipifation  of  latent  defects; 

-  detection  of  precipitating  defects  (immediately  and  with  stress  time);  ' 

-  detection  of  previously  precipitated  patent  defects  ’immediately”  due  to 

different  teat  or  environment; 

-  detection  of  previously  precipitated  patent  defects  duo  to  stress  time; 

-  detection  of  "constant  failure  rate”  defects 

For  suitheBiatical  purposes  this  can  be  reduced  to  three  distinct  terms: 

-  detection  of  previously  precipitated  latent  defects; 

-  detection  of  latent  defects  precipitating  during  ESS; 
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-  '  dataetior.  of  dafacta  praclpitating  at  a  eonatant  rata  i.a.,  datarminad  by 

tha  liaiting  NTBr. 

Tharafora,  tha  aathaaatical  nodal  can  ba  rapraaantad  byt 
OMMOVBO  -  OB  X  OPhT  ♦  DB  X  DLAT 

fl  -  aap(-kt)]  ♦  OB  X  CFR  X  t  (h-9) 


ahara  OB  ■  dataetion  aFf leianey 

ORAT  *  patant  dataeta 

Ou^T  ■  latant  datacta 

k  ■  atraas  eonatant 

t  •  atcaaa  duration 

CFB  •  eonatant  tailura  rata 

Tha  ranaining  latant  and  patant  daCact  danaity  la  givan  by: 

OKBMAIBIMG  •  (I  ",  OB}  OpaT  ♦ 
u  -  OB)  OtAT  (1  -  aap(-kt)) 

♦  OtAT  (•«P  (-kt))  (A-IO) 

40.7  Chanea  Oagactiva  Baoonantial  Modal  tCDB).  Tha  COB  nodal  la  baaad  upon  th« 
aaaunption  that  tha  population  ot  parta  within  a  lot  oC  lika  aquipnantt  ia 
eonpriaad  of  two  aubpopulationa>  i.a.,  a  nain  aubpopulation  of  "good*  parta  and  a 
nuch  anallar  aubpopulation  of  dafactivaa.  Tha  dafactivaa  contain  najor  flawa  which 
dagrada  with  atraaa  and  tina  and  ara  nanifaatad  aa  aarly-lifa  failuraa.  Tha 
failura  rata  of  a  dafactiva  part  ia  aavaral  ordara  of  nagnituda  graatar  than  tha 
failura  rata  of  a  "good”  part.  Tharafora,  ralativaly  faw  dafactivaa  can  doninata 
tha  raliability  of  tha  aquipnant  during  aarly  product  life. 

Additional  aaaunptiona,  tarna  and  dafinitiona  which  ara  uaad  in  tha  COB  nodal  ara: 

(a)  Tha  nunbar  of  dafactivaa  in  an  aquipnant  ia  Indapandant  and  identically 
distributed  and  tha  distribution  is  Binomial  with  parameters  N  and  P. 

whera: *  M  ■  total  number  of  parts  in  an  equipment 

P  ■  average  part  fraction  defective 

Por  large  B  and  small  P  tha  Binonial  can  bn  approainatad  by  the  Poisson 
distribution  so  that  0  «  BP  is  tha  average  nunbar  of  dafacts  par  iten  (defect 
danaity). 

■W 

D  =  NP  =  2;“iPi, 

i«l 
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wh«r«:  rii  >  quantity  of  part  typa  i 

Pi  ■  fraction  dafactiva  part  typa  i 

Tha  dafaet  dansity  0  ia  ona  of  thraa  paramatars  of  tha  COE  modal. 

(b)  Tha  failura  distribution  of  tha  “good*  or  main  subpopulation  of  parts  in  an 
aquipmant  ia  axponantial  aith  paramatar  Aq  and  tha  raliability  function  is 
givan  by,  Rq  (t)  "  a'^o^.  Ao  is  anothar  paramatar  of  tha  COB  modal.  The 
parameter  Aq  can  also  be  asprasaad  as  Aq  (N-0)A(2,  where  A<;  ia  tha  average 
failura  rata  of  a  "good*  part 

(c)  Tha  failura  distribution  of  a  dafactiva  part  is  asponential  with  parameter  Ap 
and  tha  raliability  function  if  given  by  Rp  *  a'Apt.  The  parameter  Ap  is 
defined  as  tha  average  failura  rate  of  a  dafactiva  part  under  a  particular 
stress  environment.  Nota  that  whan  tha  COB  modal  is  applied  to  a  screen,  (1  - 
Rp)  >  1  -  e'^Ot  ■  SS(t),  tha  screening  strength.  Rota  that  the  average 
failura  rata  of  a  defective  part  is  much  greater  than  the  average  failure  rate 
of  a  "good*  part.  i.e.  Ap  >  >  Aq  and  with  large  defect  densities  the  failure 

,  rata  of  tha  defective  population  can  be  greater  than  the  population  of 
"goods”,  i.e.  OAp  >  (N  -  D)  Aq. 

Givan  that  a  system  contains  n  dafactiva  parts,  tha  conditional  raliability  of 
the  systaa  Rg  (t/n)  ia: 

Ra  (t/n)  ■Ro(^)*RD(t)'*  n-O,  1,2... 


Using  tha  Binomial  tha  joint  probability  of  survival  and  n  defects  present  is; 
Rg(t/n)#P(n)  =  ( “)  p" 


Bor  large  N  and  small  P  tha  Binomial  can  be  approximated  by  tha  Poisson  with 
paramatar  D  >  NP  so  that  the  unconditional  survival  probability  for  any  number 
of  defects  m  is  givan  by: 


R(t)  =  R(t)  X  V 


(D)‘°e~° 

m! 


(A-ll) 

For  all  real  values  of  m 


m  a>  0 

Performing  tha  summation  in  A-11  gives  the  reliability  function: 


Rg(t)  =  RgCDe 


-D 


{A-12) 


Using  tha  assumptions  Ro(t)  •  e~^t  md  Rp(t)  ■  a~*Ot  above;  equation  A-12 
becomes : 


Rg(t)  =  exp 


-A  t-  D(1  -e 
0 


{A-13) 


The  failure  rate  for  tha  system  As(t)  is  given  by: 
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rvsuiting  in:  ^sCt)  •  Xo  OAon'^Ot  (A-14) 

Th«  pcobnbilitT  dnnaity  function  Coc  th«  systoa  is  givtn  by: 

Ct(t)  -  X.{t)  .  R,{t) 


sothat  f  U)  = 


7 


\  +DX.e"  «p  -X<-Da-e) 


.VI 


(A-15) 


Tho  sspoetsd  nunboc  of  fsilurss  for  tho  systsn  is  tin*  t  is  givsn  by: 


which  gives:  ls(T)  •  XoT  >  0(1-«'^0T)  (A-16) 

40. •  Relating  DR  to  field  reliability  end  tsilurs  rsts.  Using  the  COE  nodel  the 
reliability  and  failure  rate  of  a  systea  which  has  not  had  ESS  exposure  during 
ahnufaeture  is- given  by  equations  (A-13)  and  (A-14)  as: 


Xs(t)  -  Xq  4>  OiNlOe'^Ot 

\o  is  viewed  as  the  failure  rate  of  a  defective  under  the  field  stress  conditions 
to  which  the  systea  will  be  exposed  and  is  the  liaiting  NTBr  based  on  experience 
data. 

Given  the  saae  system  which  has  been  exposed  to  ESS  during  manufacture,  then  Dxn  is 
reduced  Or  and  the  other  model  parameters  Xq  and  Xp  have  the  same  interpretation  as 
before.  The  failure  rate  function  (equation  A-14)  both  with  and  without  an  ESS 
program  is  fllustrated  in  Fig  A-l. 
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APPENDIX  B 

Product  Reliability  Verification  Tast 


10.  Gonoral-.  A  product  roliabllity  vorificAtion  t*>t  (PRVT)  providos  a  moans  o£ 
astablishino  a  roasonabla  lovol  of  eonfidanco  that  the  outgoing  equipment  is 
adequately  free  of  defects  and  mill  achieve  the  required  reliability  in  the 
intended  application.  The  PRVT  segment  of  the  ESS  program  is  primarily  of  use  when 
the  preceding  ESS  has  been  nearly  eliminated  through  corrective  actions  that  have 
reduced  the  incoming  defect  densities  for  parts  and  manufacturing.  Since  the  PRVT 
is  part  of  the  ESS  program  the  normal  monitor  and  control  procedures  apply.  For 
simplicity,  it  is  useful  to  use  the  first  pass  PRVT  yield  as  a  reliability 
indicator. 

20.  Reference  Documents.  See  section  2 
30.  Definitions  and  acronyms.  See  section  2 
40.  General  Mathematical  Relations. 

40.1  Derivation.  The  objective  is  to  establish  a  mathematical  relationship 
between  PRVT  yield  and  field  reliability. 

Prom  appendia  A  Factory  yield*  eap  (-Dremoved)  (B-1) 

Dremoved*DS. (Dpatont  *  Dremalaittg.(l-esp(-kt))  +  CFR.t] '  (B-2) 

The  field  failure  rate  FR  is  the  defects  removed  in  a  time  interval  T  divided  by 
T;thus,FR  can  be  espressed  a^  fallows 

FR*( (Dpatent  *  Dremaining. (l-eap(-XT) )  *  CFR.T  ] )/T  (B>3) 

since  0S*1  for  the  field. 

The  failure  rate  for  latent  defects  under  field  stress  conditions  is  thus 
FR«  (  Dremaining. (l-emp<-ET) )]/T  (B-4) 

Since  the  field  and  factory  defects  are  related  by  the  stress  adjustment  factor 


Latent  Defects  at  field  stress  (B-5) 

SAF  - - 

Elseaping  Latent  Defects  at  factory  ESS  baseline  stress 

therefore,  the  field  failure  rate  due  to  latent  defects  is  related  to  the  remaining 
defects  at  factory  baseline  stress  according  to 

FR*  ISAF,  Dremaining. (l-esp(-kT))]/T  (B-6) 

The  defects  removed  during  PRVT  is  given  by 

DinfPRVT] .SS(PRVTI  thus  the  remaining  defects  as  a  function  of  the  defects  removed 
is  given  by 
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Oc«i»«ininq(PRVt  str*i*)"  Dr«mov«d(PRVT).(l-SS(PRVT) )/SS(PRVT)  (B-7) 

Sub«tituting  SS-l-«p(-l«t)  and  Dtamaining  from  aquation  B-7  givaa  tha  ralationshrp 
batwaan  fiald  failura  rata  and  dafacta  raaovad  in  PRVT 


OraanvadiPfiyT)" taxp( -kt)/( l-axp( -kt ) ) 1 . tTR.T 
Oatng  tha  ralationahip  yiald-  axp(-Oraaovad) 
givaa  tha  daairad  ralationahip 


/(SAr.{l-axp(-KT))l  (B-B) 
and  dafining  HTBr(latant)-l/PR(t-atant) 


tiald(P!WT)M*P-{l«*P<-'‘t)/tl-«P<-'‘t>>l*»  /ISA».(l-axp(-KT).itTBPl}(B-9) 


Tha  valuaa  for  SS  or  altarnativaly  tha  pracipitation  factora  for  PRVT  (k)  and  the 
fiald  (B)  can  ba  datarminad  uaing  Procedure  C. 
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APPBNDIX  C 
Fault  Coveraaa  Data 


Tablas  C.l'and  C.2  provida  fault  eovara^a  aatiaataa  for  varioua  automatic  tast 
SFitama  usad  by  alactroniea  systam  aanufaeturars.  Fault  covaraga  aatiaataa  ara 
dafinad  for  apacifie  fault  typaa,  a^.  digital  "atuck  at  1  or  0”  and  do  not 
rapraaant  tha  coaplata  fault  apactrua.  Application  uaaga  and  aituation  aanaitiva 
faulta  auat  alao  ba  conaidarad.  Thua,  tha  valuaa  providad  in  Tabla  C.l  and  C.2  ara 
a  guida  and  ahould  ba  u‘:ad  with  caution. 


Tabla  C.l  Fault  Covaraga  va  Taat  Typaa 


Laval 

Aaaaably 

Tmnt  Typ* 

Fault  Coverage 

Aaaoably 

Production  Lina  GO~NO 

GO  Taat 

0.8S 

Production  lino  In- 
Circuit  Taat 

0.90 

Bigh  Potforaanca 
Autoaatic  Toator 

0.95 

Unit 

Parforaanca 

Varification  Taat 
(PVT) 

0.90 

Unit  Factory  Chwekout 

0.95 

Final  Accaptanea  Taat 

0.98 

Syataa 

On-Lina  Parforaanca 
Monitoring  Taat 

0.90 

Syataa  Factory 

Chackout  Teat 

0.95 

- 

Cuatoaar  Final 
Acceptance  Teat 

0.99 
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Tabla  C.2  Fault  Covaraqa  for  Automatic  Test  Systems 


Circuit  Type 

Automatic  Test  System  Type 

Loaded 

Board 

Shorts 

Tester 

(LBS) 

In-Circuit 

Analyser 

(ICA) 

In-Circuit 

Tester 

(ICT) 

Functional 

Board 

Tester 

(FBT) 

Digital 

45%  to  65% 

50%  to  75% 

85%  to  94% 

90%  to  98% 

Analog 

35%  to  55% 

70%  to  92% 

90%  to  96% 

80%  to  90% 

Hybrid 

40%  to  60% 

60%  to  90% 

87%  to  94% 

83%  to  95% 

As  can  be  noted  from  the  tables,  using  only  a  Functional  Board  Tester  (FBT) 
provides  9S%  fault  coverage  but  combining  an  In-Circuit  Tester  (ICT)  with  the  FBT 
increased  coverage  to  97%  and  adding  an  In-Circuit  Analyser  (ICA)  to  the  sequence, 
increases  coverage  to  99%. 
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An  illustration  of  fault  eovora^o  for  a  saapla  of  1000  PWA's  subjsctad  to  various 
also  providsd  in  Tabla  C.'3.  Ths  stratsgiss  smployed  include  the 
use  of  each  of  four  automatic  testers  independently  and  in  combination. 


Table  C.3  Fault  Detection  for  a  1000  PCB  Lot  Sise 


^luU 

CUisiMcsttee  ActusI  US 


Sltertt  241  2(1 
Oseet  $  S 
NtSStftf 

Ceeeoeefitt  30 
Uronf 

Ceeeonsntt  (7 

tevensd 

Coeeonents  7S 
tent  IMSS  43 
Anslof 

Seecificstioflt  2S 
Oifltst  Le4«e  27 
(erfenunct  21 


ICA  tCT  UT  '  ICA-ICT  |CA-»T 


2(1  2(t  2(1  2(i  2(1 

s  $  s  s  s 

2S  21  .25  24  27 

S3  (I  SS  (4  $1 

2(  23  2S  27  28 

38  43  43  43  43 

>3  21  18  21  21 

20  27  20  27 

2(  ?( 


tCA- 

ICT. 

lC7-f8T  rst 


2(1 

2(1 

S 

$ 

27 

29 

SI 

28 

2$ 

43 

43 

21 

22 

27 

27 

2( 

2(- 

7et4l  so. 

04  Fsvltt 

$12 

2(( 

421 

4(2 

40( 

470 

497 

498 

SOI 

^4ult  C0vtr«9t 
Fsutt  Covcrift 

IOCS 

S2S 

82S 

90S 

9SS 

92S 

97S 

97S 

99S 

tecrssse 

• 

« 

• 

* 

- 

2.2S 

2.3S 

2.SS 

4  51 

leJectM  8C8s 

318 

223 

34S 

370 

385 

374 

311 

313 

3M 

■ewerk  riels 
uasttectts 

195 

31( 

354 

37( 

3(1 

384 

388 

393 

Fselty  8C8 

203 

82 

44 

22 

37 

14 

10 

S 

Aewert  rielS 
levers  riell 

49S 

79S 

89S 

94S 

91X 

9(S 

97S 

MS 

leeresse 

- 

« 

- 

• 

- 

Zt 

2.1S 

3.2S 

4. SI 

finished  Units 

lOS 

918 

95( 

978 

9(3 

98( 

990 

MS 

rae  faults  detected  are  typical  patent  defects  and  do  not  cover  the  spectrum  of 
defect  types  of  interest  in  stress  screening.  The  statistics  provided  in  the 
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tabl*  how«v.r,  provid.  a  basi.  for  davaloping  astimataa  of  dataction  afficiancy 
vhan  a  atra.a  ’.craaniag  program  is  baing  plannad.  Tha  data  should  also  ba  halpful 
in  salacting  tast  stratagias  for  usa  aith  strasa  scraans. 
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MISSION 

OF 

ROME  LABORATORY 

Rome  Laboratory  plans  and  executes  an  interdisciplinary  program  in  re- 
searchf  development,  test,  and  technology  transition  in  support  of  Air 
Force  Command,  Control,  Communications  and  Intelligence  (C  D  activities 
far  all  Air  Force  platforms.  It  also  executes  selected  acquisition  progra  s 
in  several  areas  of  expertise.  Technical  and  engineering  support  v/ithin 
areas  of  competence  is  provided  to  BSD  Program  Offices  (POs)  and  other 
BED  elements  to  perform  effective  acquisition  of  C  l  systems,  ti  addition, 
Rome  Labaratorys  technology  supports  other  AFX  Product  Divisions,  the 
Air  Force  user  community,  and  other  DOD  and  non-DOD  agencies.  Rome 
Laboratory  maintains  technical  competence  and  research  programs  in  areas 
including,  hut  not  limited  to,  communications,  command  and  control,  battle 
management,  intelligence  Uiformation  processing,  computational  sciences 
and  software  producibility,  wide  area  surveillance/sensors,  signal  proces¬ 
sing,  solid  state  sciences,  photonics,  electromagnetic  technology,  super¬ 
conductivity,  and  electronic  reliability/maintainability  and  testability. 


